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' AfiSTRACT * 

' A f'ramework^for monitoring quality control (QC) of 

processcrr contracts adininistered by ' the. Department of Education *b> 
0ffice pi Student Flnaftici^^^ (OSFA) is presented and 

applied to ttte^E^elS^ Grant Vprdgr am. Guidelines f^^ establishing QC ' 
me^^ures and standard^' are included, and the uses of- a sampling- 
procedure in liie QC system are outlined. Attention is f ocusted* oik bpth 
thf QC plan aia the-rol^ of^the contractor *in .following 'the QC 
measi^res. The processing function parfonsnbd by the contrarc^^^ the 
handling of information passed betwee^n t|ie fndancial aid app4:icant'^ 
and the OSFA, including applications firpi^ students or iiistitutions/ 
accounting reports,/ or invoices. For each cff the following areas, 
methods for develWing QC measures and. standards' are consider 
production control; fiscal contrbl; softwaire' quality assurance; QC 
reporting, monitoriQg, and corrective action; and quality of data. 
Appended are a s^t^^s- of s^mpi^ production t:ontrQl r,eports; t^ 
application 'of a fiscal control system to the Pell Grant application 
processing 'syst^; report for^iats for monitoring cost centers; a 
matrix of software Resting techniques; sample p£'6ductivity reports 
for the Pel4 ap^lifieiiop proc^^f^i and^n IMt o^^^^^ 
for a procei5)?oriQC system*' (SW) 
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* EXECUTIVE SUMMARY 



The delivery of the Education Department's (ED) financial' 
assistaxipe prcigr guns depends on the coordination of several inter- 
related parts, ^ criticai|, one being . $^he processing function---' 

the hinsiiing of information pasded "bet^i^en the applicant .and the 

■■■ • ' ■ ' ' ,' ' " * 

'office |of Sti«3ent Financial Assistance (OSFA) . The processing 

function is performed by firms on contract with ED? "therefore, 

H) must monitor the contractor ' ^ performance to ensure the effec-^ 

. ■ '" _ \ ■ ■ ■ ■ ' ■ ■ ■ . ■ . ' ... ■ , ■ ■ ■ , ♦ 

tive delivery o^ the program* A quality control QC. system with 
Measures and standards for critical activities can be used to 
lopni tor this g^^j^jpssor contr.actor and tO' iroprove -the nuyiagWent of 
the contract. This' report presents a framework for establishing 
such a QC, system and applies the framewrk th6 tq Pell applica- 
■ ^'ion. processi^ig system. ■ ;,■ .* -... . . . 

The four areas of concern for establishing QC measures and 

, . * 

starwiards- in 'processing contracts, ares •■ 

• production control 

. • Fiscal -con-trol ' " ' • ■ 

# * Software quality assui'ance • ' 

m Productivity - ' / — 

Product ion 6ontxol is the monitoring of the steps that are 
involved in -proco ssing th» in forma t ion j^oming - in 'and prod ueing. ™ 
the final output docuinent of the systeg> Fiscal control is the . 

raonitqrir^ of tlie;ffi>.urcj^s of eoftt-s v«ithin the system. A cost . 

I* ,'' ' ' . . ■ ■ * ■ - ' . 

control system routinely gathers data on the Costs by cost 



center. Software quality assurance - is .the method by vSiicK the * 

*^omputer programs' us^ in the' differ ent production steps are 

^stad* for accuracy and efficiency. Prod uctivi ty re f er s to <he 

efficiency of the labor, component in afll the manual steps in the . 

production p^rocese.' • .... ■„'>'.■■,,■ ,.' 
** ■ * ' 

A general framework is developed that ie applicable to all 
processing 'contract sv Then it is applied ^specificaily to^^^^t^ 
Pell 'application prt>ces$ing contract- Specific guidelines^re ' 
piroposed ' estaBlishing 'QC measures and standards for each area la^„^^^^^^,,*y^ 
' the process. In addition, the uses of a sampling procedure in the 
QC si^stem are outlined.. ■ . 
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I tW Office gtf stu(lent; Pittancii (OSFA) • within 



t.^e jauqa^i^n Oe|>ar,tJhfe»t - ( W o^^^ ^epyic«s to stiadents and , 
institutions thr^ugH a * numb&r of- financial assistance programs. 

^v2!|^e se;, pr*p^,i"am^ „5.'airg,e .ani3v_ cattplex* to/idminist^r. - The PelJL 
prog raiti has robre than 3 njilHon appliceirits and over 2 million 

-recipieptSr and ttie GS|j .prog^aon lias i9or« than 11,000 lenders. 



The de'l'ivery 'pf student aid programs involves several 4 inter- 
reiia,ted |>ar'ts atll vprking together in some, coordinated fashion. 

, ■ ■ , . ■ - . ■ ♦ , - , '. ■ ^ , , 

One of the largest >patfts of the delivery system, and perhaps the 
most importaiitV is- the' procfessindi f umjtion. 

- This function ' i's the liandling of information pa sse<p between 
the \ applicant and OSFAv and the transformation of that inforiaation 
int0^ a meditsn that will permit the delivery of the' se.ry ices of 
the pr^o^rsusJ In -Uie " student as^stance jarograms, this , 
information* is jusually applications from students or„ institu- 
t|ons« kccotmtinc^ reports* or tovoices. The processing, functions 
are typicality very^lCar^e, operations. They require large staffs 
,to perform several different ccxnplex tables. The prbceesii:^' 
functions are sometimes nXeo highly autcKnatedt requiring /sets of 

to send baqTc to the poteritial recipients.^ All the'raajor- . 4 

, , . • ^ , ■ ,. ' ■ ' • ' 

'procHsssi^ functions In'- tl^ stisdent *a8slst«£9e iirogra^s a^e.:done 
by eontractoi^s outside of ED> 



Processing contracts tiff er in a significant vay frcsn otJher 
jontracts in tliat they are central to the delivery of t*he student 
a\ssistance programs. . The monitoring' , of' •quali,ty control (QC) - 
piocfdures in these' contracts is critical for twb reasons: to ' 
enteure" that- the contractor is performing its function in a cost-* 
ef£^ctiv6 manner, and to jjrotect the ti seal integrity of 
prograxn by keeping spending thin the program\s "fhe 
si*2efe of these contracts,' however r make effective iaon^toring 
^x'treinely aifficult .' Methods must be establi shed ' fo reevaluating 
"the contractor *'s pferforraahce* against^ standards set prior to the 
contractus initiation. Since thsi procfessor function is usually 
<:ontracted to private firms, the process Of providing quality ^ 
control for this function has he^n characterized as vendpr/pro- ^ 

■cessbr' QC. ; . \ '■[■' 

l*his rejxjrt, suggests an overall ^^iramevorlc- for monitoring QC 
aspects of processor contracts ldit|ii>is|erad by OSPA. In the - / 

. final chapter, this frame\c>rk, is appi-i^ specifically tp the PeU 
contract, and guidelines are- r^cowiiiended. Before discussing the 
contents of th;^ report, it is nee^Sary to review «3roe background 

'^on the procfessor function in OSFA. ' ' 

BACKiSROUtm TO THE PR0bSSSIKf6 FtlNCTION ^ / 

The px'ocessing function is only ot^e of Several j^rts oflthe 
student assistance del ivsry sy Stem . 0tW par^e include ^ - 

policy detfeminationr budget ^ ^iigaxaing of aid, and 

fiscal reconciliation. " Tl^, .Shivery o^^^ student assistance is . 
dependent i^n the efficient interaction of all the parts. The 



system .is cooipl ex because the. parts of it can run either /sequen-j 

\ ■, : - . ■ ' • . ■ ' ■ V ' : ■ ; i f 

tially or simultaneously. That is, the decision, from one part is 

nei^ed before another part can begin its function^if the parts ; 

,are running sequentially. If the parts run in parallel/ they* 

follow <iiffeif6nt* tracks, at" tshe saine time with sevefal coiimion *. 

points between them vftiere information is passed frc«n oi^^unct'ion 

to ano€heir. in such an- enviroment> the system^ rtm srtibothly 

inNj^^r t^ deliver the programs ef f actively. • I.f a decision is 

delayed, for instance, the p-tiher functions inay not be able-to^ 

perform, their tasks. An obvious example of this is, the effect . 

decisions on. the Family Contribution Schedule have on the pro- ; 

/■I • . ' , ' '■ ■'■ ■ - ' 

•cessing of applicants. Without all the parameters and rules in, 

■ ^ ' ' :■ .■■ . /' / ' V^'" ' . . ■■ • :\ ' . 

place, much . of the 'fonns design, di'^sefiiination, and processing of 

-appl ideations cannot be done. ,»v ; ' 

A delay in a part of the system, can ',cau9e j>ro^^ 

contractor beyond .its control . A .change in a, procrram*s policy 

■■' ' ■ " ■ •■ . : , - - ■ .■ 

majj have significant effects on the way the processing - function ;^ 

* * . . . ■ . 

is to be per fotm^d. jlf the change is del|uyed, not fully inte- ' 
^grated iilto the j^ocessir^ systemr or' teclmically^cosplex t €het% 
the processing function may encounter prolyl eiws that cannot >e : 
overcame duri.ng the procesfiii^ cycle* , ^ ' ' 

Cfurrent Situation 

. _ _ ■„ _ ... . . ' ...... r ... [. : 

Problems with the delivery of student assistance can be 

traced to two sources: the idiosyncracies and mipredictability 

of thfe piolipymakin^ process and -the eo^ oT the ' del ivei'y 

systea itself, ^ing a' central' part pS the delivery system, the 



processing faction has :)i>een both subje^, to^nd '^ cause* of softie 
of those problems. * \ i . . 

.... . ■ ' - . . \A- -U,. 

The delivery of student asaistancf involves a[ lar9[e,riumbetl^:i^ 
of people both inside ana outside OSPA. The, deveippm^nt' of^^^^^^^^^^^^^^^^^^^^^^ 
■•pdliey— setting ; the ■parameters of ihe' progri^^-involvies ■iiiie:-'':-^^''-^-'' 
coordination o^. several OSFA divisions. Decisions iiad» outside - 

: . ■ ..V , ■ , . / ■ .:v v'-^^:^^^'"- 

the Division of l^ogr am Operations (DPO) affe6t V*i,ai ^;h^i'^^^^> 
processor will do and >^en. This; requires ext%sive <5f^ 
'tion .fi^dng - divisions thaV does not al way^*;;Oc^rli-'r.;., .tj^ 
■ is exacerbated by the fact that the ^-strijctiure., pi 

; ^ _ : . , ■ \ . ,■• . , , , . : , . : , ■ , • , . ^..-^^ V''/"' y/^;; J'^-;,; '^'^^f^: 

^iong'' functional / nqJ^Vogtam' ■ lines ■ -wi thtsfut' 'k:,«r03'e'cit ''oi-. lu^it^'li'^'*'' 



wanagjeraent striflsture. Ccsnraunication ^y \}^,'-xiio§^f'^^iM-i0tii&i- 
■^SeeaYse ' prograro.. decisions' cut . across divi^sion^;. 




The method for delivering student ^ssi«t^«^ce has i^volvfed 
with tl^B '-..changes ., ift' ^ p:?og,rants.* . .'structiirear. :'Th6rje,v,lMi's"';A^^ - ' , 

■., . ■ .■ ■ . ' --X- " 

an evfiluation at* any point ^to detemine Uf signifi^ chan^e^ in 
^ the . system are -needed. This is also true' ''^^f; tli# processing ■ 
functions. Part of the second stage of tW ciifrent ;C?via 
Control study fdcuses on the wa^ the current del ivj^y syst^^^ 
works and vftiat* changes laight improve its efficiency.. Tlie result 

of this evolutionary process is a systan that is soin^Vihat 

' . • . *• ■ , . ■ ^" ' 

fragmented with no easy \Ay to evaluate the total system's . 

^ , . ■ • ■ . ' ' . ■ . " , ' ' ' * ' ' ■ 1. 

per foxroance. "Only )fecently ii«s a systematic beqn made to 

establish quality control standards within the processing • ^ 

•contracts and to monitor their i>er f oz^ance front a manag er ia\ ^ " ♦ 
perspective.** $tandar<^^ are\ explicitly put into the contract*/ 



: : -There ^have never been significant atteanpts to improve the 
•^tt^lity control Off processor contracts, Usugully QC g^eral ' 
^ui&iSlines h?^ve 'bee.n included in the., requests for -prbpfc* sals, and 
bid4«r^ have responded with general -industrial QC standards/ The 

rfiSuit has .often«peen that the QC coinpQnents of processor con- 

' \ ^ \ . • . z , ,■ ■ . , ' , ' ' ■ . ■ 

tracts have not adi^quately addrefssed the anique requirements of 

^ . : ■ \ ' . . ' ■ ■ ■ ' 

stiident aid processing'. This report iS; an att^^^t to take "^a step 

Wck ani^ aslc in a general sense ^at OC procedures, could be 

*. " * ' ■ ' . > ' ' ^ , " . . ' . 

itnpleraetnted for ^ processor co^n \. ' ' 

' ,■ , ■ ' r / ■ T ■ - . ' . \ - . 

3:deal ,Sit.tiaticm . : " 

'. vXn ^ ideal world, all parts jOf the del.iv'ery system would 
function smoothly in both a structural and a temporal sense^ The 
system' vould be evaluate*^ periodicall^^fdf ef ficienpy^ 
allovied to evolve free4.y.. The dif^^int processes within -^be 

sy^t^ viiiouXd be evaluated to determine v^ether their config- 

■■ ■.. , • ■ : ■■■■■■ V ■ /■ ;. ■ : ' ' ■ ■. ■ ■'■ : ■ . ■ 

uration is the best suited to -deliver the services, o^'the 
program. vcmld als6.be int«^ra ted into all aspects of the 
processor function, ^y modifications to the processes would 
then b^ mad^ within established QC standafds for performance* and 
ways to measure perfortaanc^ wuld be firmly- established. The 
standards would focus ion the critical points of the processes, •. 

■ . . ■ / f \ ' ' ' 

and measuTjes v0uld be put into place to assess perfonnance 

againBt;-those:*standards.- ■ — — -^^ - — — 

In a temporal sense^f the ideal system WDuld have efficient 
coordination and .ctxumimicatioh aisoog divisioiis so that deoi- 
sions required before processing, such as those regarding policy, 



vs^ul<i bte made iii a timely' fashion. The opera t'ional managers -a 
would warK 'Closely Wth the policy manager s anci detenaine \»ihat 



feasible f?-aci both, a ^licy and arT operational sense. 
Interim solution ' , > ' 

• _ ^ ■> ^ ■ ■ 

. The ideal system does hot exist. ^An ijftteriin ,'solutrio.n to. 

V ■ ^ ■ : . ■ : ■ , • ■ > : ■ ; ' \ 

the current set of problems regarding the system and -the pro- * 

cessor function within tlie system is to establish a parallel 

■ - .,• ■■; . • , . . ^ ■ ■ • ' „■ : ■ ■ ^ ; •, 

system* of quality contr61- A formal raechanisra should be esta-- ' 
bii^sKed that would- set per*formance standards for the. proaeesing 
functiobfi and determine ftethods for meaeitring actual perfor-- 

^ . m ■ • , ■ . . ; ^ , , , 

mancel Then, an incres^ be used td srlowiy ^ 

integrate QC standards irito processSir contracts-. Over time^^^he V 
quality control ioeasures and standards cpuld become a formal part 

of the -System.'. , • . - ■ - ^. . ^ 

■ Tcj' begiij an apprdach for esstabJLlshing ' these* standards* a ^ 
general -quality control framework ' for all processing fundtipns ' 
should be dev^eloped and standards of performance eetablifihed. • 
Ways of measuring B6^^o*^^ce' should be developed' and tested. 
The mea^jres should be flexible and subject to change depending 
on 4»*iether qr^ot they are ei^^ctiv6. As the processing function 
changes, so should the measures and standards « The *m6a sure sand . 
standards s.hbuld then be applied to specific contracts tO' deter- . 
min^ their "effectiveness. "^' A^ "ED miiiagers ^ecsaioe raop6 cdinfortable 
i*ith the monitoring process, the measures and standairds may be 
ap|>li^ 'tmre extensively *to contracts, 'and as a result thfe # . ^ 
process of integrating qualicfcy control ittto the syst^ can begin. , 

■■ ■ - , ■ .■ ■ * " • " '^^ ' ' \, '\ ' ■ ' ■ • % .'■ ■ : ■■■■ 

■ ' ■ ' .■ 12 • ■ ' ' . ■ • ■ \ 



ERjO:, APPROACH " . - ■ ,. « ' •"*•'. >■ ■ 

- ^ A single framewDrlt for -establishing a method of quality 
^ ^- . *• • . . . ' . ■ 

control in processor contracts that cats , across all aspects of - 

the processing ^ function is d^sveloped in tlu.s, report,, viiose . . 

pufpose is to assist the monitor js of -the processor contr^icts in 

6 SPA- in the construction of-" a quality ' control «ydteni .for the 

contracts . .The systein sh6uld- be ased by the/ riiahager s to inoriitor 

■ . , .;,r- . , y ■ ' ■ V : . V. . ,j- / ' ' ^ ^ . .. 

these contracts nK>re' effectively. The desice<5 result is an effi- 

, ■ .; ■ ■ • , ,.■ ■ : 

cient and cost"-^ffectiy<e processing function. 

^ In an werall qu&ldty c?ontrol functiori* t%w aspects mjurtst be 

' ' . ^ ■' ' / ' * ' , " ■ 

defined: " ' ■ , ' ' ■' 

• • A- quality control pl^n * . 

. • , .Th^ respecjfeive roles of the contr^^ 

This report focuses ori bpth the quality control pi§un and th%5 

rol45i^9f the contractor in eSfcdn^ that quality control measures 

are taHen. The role of OSPA is one of quality assigpi^ce, invplv- 

^> ■ " - , , ■ ■ ^ \ ,^ 

ing tnonitidring the quality control lecKJedur'es necess^y iBbr 
effectiveness • The issue of ' ^liality assuran^ vdll be the topic 
of a secom^ rejKJrt oil yerKi or /processor QC. ^ . ^ , r 

ft-quality control plan can I^Xestablished for all , procetisiio 
cbntractfi within OSPA# . The first step is^to i^SentiVfy how 'the \ 



processor functions ^are simila:p. /Although the J^rocessit^ f ^duic- ^ 

process point of ; v lev they- all c6j|taih certain st^ps at^ proce- 
a^e8'.<^ Pbr- instance, ■', at i'eafft a ■;^rt of ■''ill ; fp[i.ctioiiS' ar«j_ 
automate^:* Therefore^/ in order tQ ensure that tH^ computer* 



programs are performing their function© accurately a^d . 
efficiently, thi?re must be some coaaAion. vay to test them,. ». 

Once the functions' ' commonalities have been identified, a 
set of ^neasvares »n(3 standards can determined > rtodified^ and 
applied to the particul^r-^ processing contracts The f 4rBt • step in 
establishing the standards and' measures is to identify the foi- 
loving or iticail areas that require quality control procedures. ^ 



m , Prbduction Control — The steps of the production . ' 

. process require monitoring for effieijpncy and - " 

accuracy. , ' 

■ ■ - < . ■ . ' . * • 

Fiscal Control -"-Costs of the processing function must 
be closely monitored to ensure, that the contract 
remains within its budget. • 

software — ^The ccataputer programs that compute vaitie^ or 
j>rocess information idiich must be accurate and 
efficient, should be thoroughly checked and \ . 
continuously monitored because^ of th^ir complexity. 

Productivity "— .SCHtie of the\jsteps in the processing * 
.f unctiotf «are .manual ? since labo^ costs are high an - 
ef f icient mix of labor and machine must be ' 
determined V • ■ ' ' ■ ■ 

• • ■ , ■ . ;• , ■ '.• ■ >r " ■ " , ■ •..!'■ , . 

QC Reports — ^The "iaanag er 6 of the contract mu st r ece xv e 
the pertinent information from the 'processing function 
In oii^det- to' %:&€ectively Bto^ aspects of the 

■ .dbn tract. ■ ■. ^ , - 

•■ ■ . • ■ ' . " ■.-'■4, ■ - . ' ■ . , ■ <■■' 

'■ ■ ■ ■ ■ . 

Corrective Action — 't f the pr6du6tion process re<tuire« 
cbrfngej '"^here must' be 'a mechanisa. for effecting it 
that minimizes dislocations |ind time Relays ih th# 
• process. ■,, . ■ i , ■ , ■ • ' . " ' 

' .1 ' ^ , • ■'■ . ' ■ ' " ^' 

Quality <>^-1>atar '"*nv^ process mu&t g^anerats f 

,^ccura te ir? formation; or if additional^ work . or/' 

■ correctiohfe aire- n^ed^d to rectify errors, th0 prpces'a 
■is ineffijCi.eD$.,,.i^_.'*- • ' . _ ' " '"^ ; ■-/';,; 

Rework , ' an<i ' Re<$andancy in Pro^uctidft Proeesaes -^ f "th^ 
production ^tepa are not sufficiently designfed, 
-^nef f icienci^fl and unnecesBary backlogs could ocfcur as 

■ ■■ a. 'result ,o£. rfeduix4an!Dy.- / , ' . 



^fitl oping' 



» In eacK of tHese area Sf raettiods ftfr devfid oping ' QGilfee 

aha' -siandards are ccftifilder-ed, ai^ particulat steps can be identi-i 

'• ' ' - , - , ■ ' . ' " ' ' . . ■ ' .■ «. -. ' ■ 

' f-ied 4s ^0ing amenaiile to quantitative meaisureonent. The remairi-" 

•• '. ^ ■ ' \ . • ■, . *. , ■ , ■ ^ /. _ * ; " ■ . ' ; . * . 

^<aer of this rei>ort Ipoks at different aspects of the pr pees s'i^g ' 
V function and ^identifies areaa that should bf 'rapni tor ed. Fox: .eacli' 
^ a^^ea a m0thodolp/y for establishing; measures 'and standards is 
/ developed. ' The €£nj«LL phapljer illustrates the utility of applying 
/ fhis'rfraraework to t^e Pell process. . • 

Each' of the fitst four areas x^epresents particular corapo^ 

aeniis of the procissing function and is dealt ^th explictly in^ 

" ' ' i' , ' ■ *i ■ ' — ' ' • .■' . ■ ■■ ■ ? _ . ■ ■ . ' ' ' , * ■ 
its pwfa qhiapter. ° The latter *fe>ur are critical areas that trans- 
• ' * ■ . ■ ■ ■. s ■■■■ ' ' ' ' • ■■' 

^ cend particular areas of the prbcesyp and need to be addreefted by 

ttieuiager 9 ipt any of titer proce§sirtb[ far eas* QC report:: s and eoj^- 

rectlve aetdonfi are managmeat tpc^s, use^ that the 

pVoducfcion process is rmifiing smoothlyr Quality of data and 

, re-work are. are^s that need feb be cpnstantly tested for so vthat 

cprrectiv enactions may b^yquiclci 
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• ■ CHAPTER 2 , ' " - . ^ 

' » ' ' • '■" \ ■ " '■ *■ ■ ■■■■ \ • ^ 

•u .' • . \ PKODUCTION CONTROL ' ^ . 

^: The j>rocessing function in the delivery of student, assis- V 
tance is cleared 'to\rfatd j^oducing a product. That is, t^e|. f uric- 
trioti obtains a g^at deal of* data, processes it/ transforms it 
into a set of infor«iation needed to deliver the service* and 

. produces ' an output document. For ej^^pleV the primaity processing 
tasX of the- Pell application processor is to |>roduce a Student^ 
Aid Report (SAR). Other output docuajents, such as applicant, ros- v 
ters and, state agency data tapes, are also produced'. But the SAR 
is the major output. ' 
-To ensure its ^integrity I the production ptocess should be . 
mp^jitored at. eatlj step JLn the process. This monitoring activity, 
ia'laiown as fproduction control. Measures to reflect the actual ♦ 
perfo^Tnance .of the activi.ty should be determined, and standards 

■'■f©r, perf onakncei ■ at' '6ach ' ertep' .shcjuld be established, Each 6tep.;, 
wili include di^i^erent ijypes of activitiess sCroe are manual, 
others ar^ " purely autoisatic/ and some are combined C^aftn^i and " 
automated!). The measures and standards associated. with each step 

'differ in nature depehding.on the Kinds of activities . required t» 

^ ' \ ■ ■ h ' ■ . . ■ •■■ •■ 

complete the step. . ' . . - 

Tbfe.:prQepaa|.t^ functiQOfi in deXlygry ■^^!BtfeiM„j9.£.;..:'ySe-.-ftt^\i'<3enfe^,.> 
assistance programa differ according to whii^t data are being pro- 
c€ts«ed^ vho provides the data, aifcd' Who tl^es the output documejats 



pr^odac^^. one can identify general steps in the 

prbcessor f Venetians . • The first. st9|> iP designing measures and 
8tan<Sa;ra8 for productiq'n control* of^^he^procesaor functions 
should be to identify the steps within the production process., 
Pigurja 1 shows tUe. following ste|>S in the- processing function of 
the student assistdnce programs! ' - ' 

" ' ■ v\'v>:';.:':>'^:^: •■.'>•::' 

• Receipt of Inf orwation- f.The point wh^^re the jnforma- 
tion f^^can the applicant is' received and ' sorted into • , 
homogeneous groups for processing. In the Pell pro- 
grajcn application pr:ocess this step is the point where 
applications are collected and put into batches £6r . 
initial processing.- In the Carapup-based application 
system, FISAPs are received and put into batches sepa- 
rating the ihit.ial application from those being 
returned for corrections. ■{ 

Data Entry -^The process by Which raw data from* paiper 
forms are Key entered" into the computer system. This : 
is the initial data transformation step of the pr6*- • 
^essv ^For the Pell program it also includes' entering 
a^plic^nj: information or SARS received from ty? insti- 
tution into the PIMS* In th^case of the Pll^insti- 
tutiomfe can supply SAHs on magnetic ta|>es, although ? 
only a few Institutions have^elec^ted this method of 
data' transmission. RegardldSb of the entry inethod 
Ce.g., keypunch, on-line key entry) « it is necessary 
to enter data- in the systemiat certain points. • 

Data^Edit-^'The £>rpcess by which datf from forms • y 
. entered into the eyst€te arte checked for accuracy and 
consistency. This step is a* more generic production 
step than some of the others 'becaue^ it can i)ccur ^ 
different, poihts in the process, sometimes more than 
onee. It can be done manually or automatically. It 
can also be folde;! into another production- step. In 
tfie Pell application process, an application goes 
through an initial manual editing process that checks 
for a "few -conditions i such as signature) that ^oald 
preclude the application from further pjrocessing . 

^ Confute— The actual processing of\he raw data is 
tram^orraed into the type of information needed t.o 
deliver the program.. This step is usually an aatoma- 
tic process performed by coi^puter. In the dSL 



iiiterest billing 4^s-bem, lenders * regueBts for payment 
of interest on their portfolios Vof in-school loans ate 
processed and an amount csomputep. In the Campus-hased 
system/ the institutional allocations are determined. ' 

- • Document Production and Mailing — Each processor 
produces a primary document that is sent to the 
, ■ ' applicant- The doctiment may be. a notification of 

awardr which confirms the -claim of the recipient or 
lists errors in application. This fiftep includes the 
# • production of the document containing the correct * * ' 
:, '. informs^tibn and the , transmittal of that document . to 
the recipient; In Pell, the document would be tlie 
SASr? in Cait^us-bafijed the document would be the award 
■ ■ letter . ' . • 

\ '\.m-. V Corrections Proce dure s- r -The procedures for handling 
C intonsistericies or abnormdlities in the production 
* • ^ :;process . "This stepf like the data edit step, is a 

. • lireneric production activity. ^Different systems handle 
it in different ways at difeferent times. In most 
cases, the corrections process, deals with corrections 

* to the information that comes into the processing 

* ; function. These corrections are sc»netimBs requested by 
i the system,- which is part of this gtep/ or they are 

' externally motivated* In Pell application processing, 

the applicant is Notified of any corrections required > 
- • : and allowed to submit* unsolicited corrections.^ Then, 

■ / ; • when cor.r'eckion^ are submitted, the applicant ''s record 
must be uj^a|^d aM- perbaps a new SAR coo^uted. ; * 

While these ■major steps ^?oveir_. the 8pectjrum,,of ::;^5?tivitie6 in. -.the 
processing functions , each has its owr^ sequence ojE steps. For a 
de^er analy$i»v':of 'thev;^^ each of •''•ti^e; '.prpcc^fi sing, function*/ 

one may r^fet^^'^o Jyavanc^d .technology's Comparability Report, June 
1982.- ' - , , , .. ■. ■ , _ 



MEASURES AND' STMH3ARDS ; - 



The tw6 maaor^^^^p 'of produi^tion control aire tds 

• Bhsure maximum efficiency of each prbductioi st^p . 

• ^ •;• r Highlight; ' desolations* ifom , pi^e-^stabliehe^S -ptodpctidn, 
.■ t--,';' '.standards''- " \ ' ' ^ ■ ■', . ■■'-••■■■ '■ . • . 
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Given these objectives, the first in e stab pro~ 

difction control coajponent Of porocessor QC procedures should be to 

■ ■'. ■ - v ■■: •■■ ■ ■ . ■•■ . '' r;'v ' , • • * ' ■ ' y 

establish measures 4[nd standards f^ in the production 

process. |^i:tical points in each Itep of the processing fmi^ibn 

should be identified and- a >et^ 6^^ 

oped &>r monitoring the actiV|;tife)6rrequired to coraiplete the step. 

: ■ . , ■■■■ ■ ■ P-:. ■ ^'-^ • ' ■ := / ' 

Since each pro,ces8tng. f iiwct$i?ii tiaB its own structure for these 

steps/ only general types 6f iheasu^ and standards can be' iden- 

tified in this frameworH. !^ applied to' 

specific prodactiop steJ>aT^^ illustrated in the final chapter of 

this 'report. V , ^ ■ . i.v:? . ■ ■ 

rhfomtttion Receipt ■ ■v-'', '- 

' The objfectives of this step ar^ to receive tihe in format ion 

S arid to transfotra the incproing document- into a form that can be 
used in the processing streanf. First/ this requires grouping the 
dijfferent ty^s of incotaing doctaa^nt»**into ui^il^t type© of docu- 
ment^. The crucial points in thlB step' are the initial screening 
of the infonaaition and the group4,ng , together. T^ierefiore, the 
activities that should be meftsured ares 

# How much information is ccsning in 
■ ■ ■ m How fast is' it being prepared for further processing 

, .«t-,_._.._?te>.w ..*»ii.Afee-)i^ . . 

The standards for this step should establish toleran<:jes for 
the timing and accuracy of^ these activities. This step is toaong 
the most time-Consianing of the steps in the production process,; 



especially iih en its design is not well conceived. Accuracy • 
shcmld not be sacrificed for timeliness, since an -error will 
/aus^ a slowdfiivin and a retracing^* of a Atuotoer of actions. Con- 
/versfeiy, accuracy should not be achieved at any cost. A wipsll- 
designed system should toe both quick arid acc^rate. Therefore, i^t 
is important to develop measures and stand ardsv for both timing 



& 



arua accuracy>vviiich should includes - ' , 

• The turnaround time for ^the entire step^ fir oro the time 
information comes in to theptiine it is ready for ^ 

'.■further .processing ■ . 

• The accuracy of separating ' the information into the 
. . cp?-rect groups for processing ' / , 

\ • Sampling, y/hich can be used as a measurement mec^tanlsm 
\ ; for both standards 



•Furthek^ specif ic^^^ncoraing documents should be identif iid by the 



• V 



\ . .. . 

contracted for each t;ype of document # and time of entry should be 
noted . The documents )^ be rechecked at th^ ■ end of t>ke 

production step for timeliness* an^ accuracy of grouping. All 
docisnents can be dated in this >ay so a ^^ple can be drawn at 
any activity point in this st'fejp'. / ^ 

Data Entry ■ \^ 

Oata entry is »usually a manual process that* is relativ;Bly 
tiitfe-con^uming . the e^ try of data fr era any type of form that 
contains a large .number of data items i^ coovplex and error^prone. 

run the processing system, accuracy is of utmost importance. If 
th^ data entry is inaCcuratte, an error will o^eur at fiome point 
in the process; usually further reworking or correction' within 
the production j^ocess will be ne&3ed. i \ 
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Tlie deve 1 opine ht' of measures and standards for thi* step 
shpuid focua, on accuracy, industry standards . for this type of 
data entry should be used in the* evaluation of the actual produc- 
tion process, specif iying units of jaeasures is critical, since 



etandards'can be applied to keystroHes,, Sata elemehts , ' or entire 
forms. Moreover, data entry errors should ^be classified by types 
of da^a elements, depending on the units pf meas^iremeht used. In 
€he complex cases? judgment is required, since sosne data . elements 
are crtvcxal to further processing \i^ile others-'^re le^ s" important 
and can be easily rectified. In addi-tiotv, standards of time- 
'liness 'should be bal-anced to ensure maximum accuracy, and effibi- 
ency* T^e^^implest way to^ obtain measures of thfese steps is to 
take^ a sample of the data input doc*uments and compare the trans- 
formed data >ir1lth the raw data. * ^ 

Data Edit .• ■ . . . 

This step can be performed more than once in the production 

■ . ,* . • . . * ' ■ ■ ■ % ■ 

... ■ ' - , . ■ ■ , ' . ' , ^ . " -. ^ ■ ' ^ ' 

profess. .^^ither iBarmally or automatically. Th^ objectives of 

' ' ' * 

this step are^to ensure that: , - 

, , -f , > ■ ■- ^ . ■ • > . , 

■»' ■ ' ' , , ^ , '~ ■ , ' ■ ' 

• Incorrect <^ta are caught for the right reasons < ■ 

♦ ' ' ' .» " 

• The right responses are sent to the recipient 
'Portions of the activities contained under data editing can be 
subsumed under other ^teps- For instance., when data are 
i^gsived, cursor^ edits ~of "pa the ~ 
document flli9ged|pr special handling. Measures and standards , 
ee^^ be deveJ,ope^' for use whenever a data edi step is 
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per'fo^ed in tlie pfocesiing function. Such taeasuree shibuld 

emphasize; V . ^ 

■ * ^ , .. < ' ■ ' ♦' ■'■■'* 

' ^ Frequency wit3> vftiich the editing procedures are invoked 
• Correctness of the system's re spj^nse to the recipient 
»' Accuracy in the total editing process 
These laeasures are redefined as follows. First, , the 
frequency should measure the odcurrence of each particular edit- 

, " - ' » 

ing error (or those deemed particularly important) or possibly 
groups ot criti^l editing errors. This c^h be done aut<wiatic- 
ally if the edit is. computerized or by ^ manual tally -sheet if 
the edit is manual. ^ Second, there 'Should be a measure of cor- 
reetnese of the .edits* Are the edits flagging the correct e^rrors 
ini tile prpp6«r way? Sample cases can beJ used for this mtfasxire, or 
a rank OKI selection of live cases kno vox by some other criteria to 
contain Incprreet ctata it«sii|| should be u«ed to test the edits. 
Third, J.t is Important to measure the accttf.aey of the respOnaes. 
This measure pertains most specifically to the manual editing/ 
^Jtere a form letter or some other device is used. However, it 
can also. IstpplV in; a. more general way to computer edits* A randpra 
sample of^^^fe^S^^S^ toe flagged to provide a general measure of 
the accuracy of the procedure. Lastly, a measOre of the turn- ' 
around time should be uded so that corrective time can be mini- 
mia»d . ' Ihi s* mea stir e' can be ' a par t of the ov er ail "tiwel iJJe s s 
measure of the production steps. 
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. Compute . , ■,. _ . ' , ■ . ' . , 

This step is almost always automatic r therefore,! the njain 
qbncern should be whether the coojputatione necessary for -Uie 
accurate delivery of the program have been per foinned. An output, 
dbcoinent is tVi© end result' of this step* and it miist contain 
accurate information. ' * 

The measures and standards for this step overlap with \softir 

ware quality assurance. The software quality assumes proceoures 

■■■■ \-. ■ ■ • ■■ ;' .. i , „ , ■ , . ' / '■ ■ ■ _ %\ ; \ , 

discussed in the next chapter/ which should ensure that th6 co^m- 

putatiohs are cor recti. Sin6e the central processors handle a, 

-.-v,--^' ... . ' J - . : , ■ 

l&rge^rttnc^t of .inf9r3nation, inefficient manual procedures in 

a f^rirtary cojiiputing eyyteinjnay be harmful to tjie overetll pro- 

Auction- Although the cc»nputation ste^ inay be tliS* least time- 

constsming, timeliness standards should be set to a 13^ hianagiers 

'..,■■<.■ . ■ ■ ■ ■ , ^ . , ■ ■ ■ " ■ . . 

• Compare current perfarmance to established standards 

and historical perfotmance . ' 

• ' Compare the time line b& of the step with other produc- 
. ' .*tioh steps *. 

Another measure for this step is a count of the nui^r and 

type of computes done for a particular^ tiR^ periods Alone it may 

not be a very useful measure, but in comparison witj^i production 

statistics from other steps it may signal problems within the * 

coAjptite/^iep .;' ~ •.. " ."• ~*"' ..""^^ 

■ ■ • • .,■ ' ' , • , ' 
Document Produc t ion and Mailing 

■ ' ■ '■ " V ■ ... ' ,'•'*■■•■.■ ■ ' \ 

I This Step is the main resiilt of the processii^ system, one 

■ ■ ' . ^ ■ • , . ' ■ ,.' 
which usually includes both auLtomatic and manual procedures.- 
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, ^ -Accuracy— in the sense of ^setnding out the correct document ( s) \ to 

^ the corr^t applicant— and timeliness of pr<^uction is crucial. 

The development of measures and standards for ij^ls step 

should focus on the cor^^ness of the output do'cuments and oth^r 

. information .;sent' to'- ^'^^^'^^^W^^^^'. Th^. -applicant should , have 

oomplete information to^miraizelthe pumber of transactions 

wit)iih the system/ Standards set for the tim&lihesi^ of 

•ithe production and mailing or coimauriicating the results. • The 

i crucial measures are: ■ ■ ' , . ' /, ' 

' • First s the primary measure for this step should be the^ 

couttt of ' output documents, produced by the system. ^ 
This measure should ise used in conjunction with cotiiits 
.: f^om other production steps to ensure the^ integrity of 
t^he system and that nothing is being lost. ' Control 
totals across steps should be maintained on an ongoiflg 
^ ^ baais l^ .. ■ ■/ .ji;,- ' ; ; 

# Second , measures of accuracy of this production zmd 
/ mailing process fifhould be taken. Random samples of . 

' % output, document^ immediately before they are to be 

' sent out should be checked for the correct documents, 

etc. 



Thigjd f aa a ^rt of the overall timeliness of the pro- 
duetion process/ the turnatound for this step sfhouid 
be/measured. ^ 



Cor rectioiia/ -Procedures. ' - 

\ This ^tep, like dkta editi is a generic step ^hat may be 

perffeirmed sWeral times throughout the processing cycle, depend- 

/ ■ ■ 

ing^on the ^stem and the^type of correction. Corrections 
should be handled efficiently -and -accurately -to minimi^ the 
nuflflber of trlansactions witltin the system. This step :fs often 
cabined wifh_the data eai^ or cpmptite steps. Howeveis,. once a 
correction Request 1ie«s been sent out, the system must jbe able to 
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handle the correction vhen it . comes back < Individual records 
, . have' to be updatedi information recompated, and new output 
doeuinents produced. ^ ^ 

the Standards for tlvese apasures shQuld tie focused on ' ^ 

f' • • ■ ' ' ■• • ■ • " 

' reducing both the time of making a correctioh and the nwnber of 
times an individual's record mus^ -be corrected. Once a-cdrrec- 
tion is received, how long does lit take to' get put l?ack " into the 
. system and a new outpat document \ produced? Since correction 

procedure standards almost parallel the standards for the rest of 
the system, the timing should be less. Alsoi error resolution « 
/ standards should be^set up so that^ efficl^t ways to resolve 
, errors are in place and vork properly. Measures for this step 

; ■ inciuiaes ■ ■ ' ;":v-:; - l \ 

• Timeliness- of the corrections put into the system once * 
they are deemed oorrect '^■^'^'''.■^^'''■■^ '■■.^y. r'.X ' ^'■■■}' 

\_ , Types of errpr resolution procedures including fre- 

quency counts of types of error resolution .procedures 

• ■' Tiffle necessary for flagging j oi^yrection to go' back ' 

out to the recipient for further- correction 

■ ■•■ ■ ■ The, ef ficienoy and effeetivenssa of resolu^on'' 

- ■ • \ , procedurea" : ■ ' ^' ■ \ • , ■ \ 

' , ; ■ „ ■ ^ ■ \ ^ .■ ■ , 

ESTABLISHING A PRODUCTION MONITORING SYSTEM 
^ The pi^rposes of establishing the performances and measures 

- standard^J4l^^^^ tl»^mtasi.J3J^-C^ . ' . 
•/ effectively', .the prodlletion process. - Th^re are >two ways such 

monitoring dan assist managers, ;, ;3?irst'*. ,- it checks the current ' .' - ' 
status of the process. This is easiest when there is a single 
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figure i»> eacb step offa .|»redefiaed set of figuresi^ for^^e 
jptep. Serond, ; nionitAring i(ldicates ^to managers t"bat sane part qf 
%he |»roces8 is not performing up to the established standard or a 
jigfend. is dev^lotyipg that Wbuld. if not chedked , result in a sub- 
Standard situation*. ' Briefly/ a «et, of -canparative statistics {or 
standards ) can be., used to produce tlie 'information needed to alert 
managers of production prdblems . The irteps ; required to accom- 
■ -■ plish' this ar^t<^•■y^^ ■ •• > • ' . ^ 

• Identify a number <>f indicators crucial jto good pro- 

* duction ^control / " These 'may«^nclude all *tJie nfeasures- >lt 

' ' . identified in tbe previous section or a subsjimple? or, 
particular n«jasji«fs may be chosen ^ reported in a 
regular faihidnwKej:©^ Other measures may be used on an 
ad hoc or as*- needed basis." *■ ^ ■ I 

' ' ' • Establish i>r6duction standards in the system by . 

^■.„',v: -putting-, them on reports' 'M »tabdardsV;'; ^ _ " , . ;;■ 

^' liert the n»asures desired'into the system' J>y" . 

creating measurement n^chanisms to routinely collect , 
' store, and r«t^ieyet the«e data for future comparative 

: v'^;-.' ''-piarposes;v , /-^ ■ ^ ' :': - , , ' 

js Construct the correct compjlr^lilve itaiistics which 

'would be uiiefui.-' ^' ■ : , \\\ V li'-. ' ■ 

■ ■■ ■ ■ ■ ■• , v " 

It may be possible to 'coiidtruct j»airticular staltistics , such 
as a^eraiges, that We aUtomaticaULy reeom^tetd ^fter additional 
data are <ft>tained. These data c*n either be kept a« of th* 

subsystem in Which they are collected or be sept to a qentralixed 

'"'(■' '«» ■ 

.J-ocation Where retrieval would not disrupt the system. The^ 

. entire-eyetem can -then be Updated on ^ A 

tion statistics >rould be, a ueeful management tool.^ 

* The repoirting fd-matrfor errors |^ the- production procei 
is one Of the most impbrtant aspect^ of psI'Ay^ control of 




inionii;oring , especially cif the processor f unct4.<5p. Tlie . prdceiis 
can collect a mountain of data. Tl^ete-f ore , care should be takeri 
with the vol \ajfle and types of reports 'produced for OSFA. Report^ 
should be Concise and aover areas c|--ucial to 'Effective . 

ifianagement by OSPA, wil;h emphasis on' aireasiof kiibstandard % ' 
.;per£oniianqe» . ■' ' / . • ■ ■ " 

The reports should W geared €o p&rticialar levels of mana~. 
gere in the OSFA. Daily or weeXly production statistics by step 
are fiot nectssary for top raanagersr aggregatedi data oyer, several 
Steps can be accumulated for them. . If exceptional cases at 
Specific step are highlighted/ th^ information can be^ passed- - 

along ' quiclcly ' ^ \- ■' ' ' • . ' * 

'-Once the pertiiient items, have been identified, the reports 
should be produced with data from different tin^ periods for com- 

parison pCirposesl For instance, frequencies of edit checlcs over 

, ■ " ' ' '■ ■ ■, V • . 

the most (recent week* an average over the last month, and the 
total to date -can be more lhforB»tive than just the weekly and . 
yearly totals. 'Statistics of trends arVas important as cumula- 
, tive . statistics ; ; Also, _ if gos&ibl6, fr^uency distributions . . / 
ratier. thai> averages, or in addfltion to average^, should be pre^ 
sentjedg Averaifes can^ide treijds that may be occurring- or out- 
lying cases that sWould be ide^tifie<i» . ' 

'■'"'".'•if -a'^tre^^ ~_a'.'stati&ti<i''"'faliB^^^^^ 
it should be ' highlighted eithir with^ a '^flag or in a summary ^ 
* report •■ Xf put intc? summary terms , along «ritt! an explaiiation, 
OSFA contract managers can glance at the report and obtain a 
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great (JeaJ. of inforn^tion about ar6as of inmjedtate cotjcern. Otie 
disadvantage of a suajmaVy of exception* ' is 'that the project 
ftianager tuay become t.oo teliant on the Bunimary and not on the 
report itself . The. entire tepojJ-t, if done veil, , should be of 
interest to the OSPA mahager. . ' > 
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■ . FISCAL CONTROL ' * . / 

Pstabllshroent of fiscal control is a key elenierit of . the ^ 

overaljt ' contractor quality control system for the processor fwnq- 

tion>' ^^FA needs assurance that its ven(3ors are perf 6rming. in as 

'cost-effective a' ma.nher;as- possible: without sacrificing quality. 

This ^chapte^j detail^ a methOfaology for estajtilTishing contractor * 

fiscal controls^ The lev61 of fiscal mondb toyitM should /.vary bjf^x 

Type of contract ( i ^^e . , cost-plus-fixf d-fee £CPFF3 . ^ 
' contracts need different toonitoring fiSiQpj^that of , 

\- , fixed-price contrsCcts) . 

• Size of contract (it may not ^ cost-^eff^ctilve to use \ 
- on-site ^onitprs for sinall contracts) ■ ^ 

^ ' (s Previous OSPA experlenpB with the contract V 

Fiscal control should be established individually on a 

contract-b]j^-contract basis I* for the reasons outlined aboii^e, dijd ^ 

should be jointly established by OSPA and the contractor. In ^ 

'Itliis case, cpntrols Wovil<? b^ less burdenspme to the QOttt^a^||5>r» 

^o| exampj^ei if QSFA required biweeX^y report iiKf but the^9;iitrac- 

• tor ,accuimilated^ co#ts on a /iemimcpthly basis i thet contJCAetor 

would have to m^iintain ah extra reporting stream just f or %his 

pSFA contract. The fpllovihg steps , are r'ecjuir^d to establish 

fiscal controlst - 

^Qevel'djp'' fiscal''' 'contra . ' " 

- Identif^ma jo^ cost cepters for measurement / 

"■" ■ ^" " .< ' C i4ienl " ■ . : ^- , ' ■ 1 "V . ,■'"'. 

- ' Select specific typeg of costs to " measure I what } 

- /vjDetes'mine methodology x>f cost ineasurement (how) 
, - Identify the unit of measure or standard^ 

' CWher^). ' ' ■ 
, Establish toler^nee ieveis 'Chow tauchl ^ / 



• specify guidelines applicable) • 

- Require response from contractor • • 

, Evaluate response ' ' ^ 

«- If applicable, modify guidelines 

' ■ ■ ■ '. ' • * ■ ' , " . ' ' '■ ■'".»',' 

Develop fiscal control systep jointly with contractor. 

— , Flex iljle budgets for low , high , and average 

■ ' ' ' V , voiurae ' , ; . 

, , * Reporting taechanisms and formats - " • 

- Specific agreenient on guidelines developed above 

• ' • Develop guidelines for PSFA-.Jftonitos'ing . 

• ■ • ' ' • ' ■ ' ■ ■ '■ • ' ■' *■ ■ ."^ .■ 

- On-'Site auditing , 

- OSPA management role r 

, - Reconciliation of repoiJ'ts to contract invoice 

. . • ' ' . ■" ■ '"'^ ■' ' ' 

The- relationship between these steps is illustrated in Figure 

2. The ste|>s are detailed a« follows. , _ . 

\ . ■ . . ■ ' ■ ■ 

j> - . . • . , . , 

• This section' jjresfin^ fiscal control sys- 

teto approach *aj^pii,e eost-plus^-fixed-f ee contracts • pbVi* 

•^slyf ' the role of 'fiscal controls in^Vendor quality control •' 
diminishes in reiationtship to the .dollar yolume of the contaJact. 
For- example, a $l60»000 fixed-price contract should ^.not require 
f iBca£ controls "of, the typ6 outlined 'here . However . the aeiho- 
dologfy detailed in this section could be applied to any contractv. 
The* steps required to develop sound fiscal controls are put liziied 
.below.. • . ', • ■■' ,^ ■ ; ■ ^ ' 

— Pix-gt J ^astr experlsntre -^ould play an ia^K^rtant role Ari^ - - 

.developing fiscal control guidelines for, any OSFA contract going 

. out for bid V Fiscal cofetrPls should be establi^dyi for oil Bjaior 
work ar^as > or cost center's , with 6verrui\^cost projections in the 

'■past.. ; ■'Fo^r' ■ etx^ple, ^'it /may. be 'necess-ary : .to establish ."ti^^^t. 
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' FIGURE 2 
FISCAL CONTROL SYSTEM WORKPLAN 



Federal controls for types of posts that have proven difficult to 

■ ' ; V . - r • ' ' ■ ' : ' ■ ' ■ ■ V' ■ , '^^ ' 

control, such as coa^uter time. However ^ it may not- be necessary 
to use the same controls to monitor relatively iinoiaterial cosCs* 
such as reproductigin costs. Identif ^.cation of major cpst ce|»t^i^« 

.•and-- typee .of costs Jto •raeasur.e' should;. ,ai:so^;;l3^;,;.t)a 
' . ' •"• Relationship o'f co6't»;'cie!nt;^r',:.to:'vv^ 

-:• ■ Relationship 'of 'cp6ts~^0f;^;;:the -'^^e^ to... ,th;6"^ total '^,i?p^t ■' '\ 

: ' m ' Inability 'to ,deVeio;p' '<^n^r6Is, „in this dr, .other ' OS PA-' 

contracts ^^ ;;^\ \./4: :'/v-/'-^;' - ' -iv'- ; 

'the Second step t qn<Je 'co estimated ,1^^^ 

to establish routing c^^portihg of all aasbciated cofits Cdiir^t' ' 
labor cps'ts) to matiagemient and reporting of deviations foi: - 
established This is . currently done by ^ 

otheir government agencies and may be useful where no contraat - 

^' y •■; " " ^' ■ : ■ .\" ■ ' • ■ ' ■ ^ ■ ' 

hist6;;ry exists. For. example! Housing tirban Development (HUD) 
uses this procedure Jor some processor contracts. Alternatively, 
direct (or "loaded") labor costs for all (?ost centers in excess : x 
of a certain perceiftage of the contreict should be singled out . 

Third* after identifying n^jor »cost centers and specific ^ . 
costs, a methddfOXogy of cost measurement should b^ determinerd. 
Basically, there axe three options: , • 

• " ^ers6n-hours (labor only) " il I 

/ • Direct costs (labor plus other direct posts) 

.# ' loaded direct costs (incXuding fringes, overhead, 

■ ■ • ; .eto-.) ■ ^ v^ ■ 

llhis decision should be. based on the nature of the procurement . 
Person-hours shop Id only be selected if viabor is a major cost 



(more than 60 percent, of the total cost) . Time and materia,?. ' 
contracts s]iou£d>e monitored only by using, loaded costs, since 



tliat is how the contract vill he billed. In general, loaded^ ' 
direct costs shojOd not bemused unless satisfactory monitorin^~"> 
can be obtained via one of the other i^ethods. > , . ■ 
Fourth, establishment ;bf tolerance levels must be con- % 
sidered . In general , an acceptable tolerance level would be a 10 
percent viuriance around the standard unless experience shows that 
costs fluctuate more than that on a monthly -basis » A tolerat^ce 
level of less thaj^ 10 percent- would generate a needlessly high 
volume of exception reporting. The tolerance level (8tat>dard) 
should be applied to- each ma jor cost center. It is important to 
remembeir that this is the mechanism ^or early wariiing- bh coiitr act , 
"overruns*.' * '\ " ' 

SPECiFY GPIP^INES ,;^ ■■' /',.;V^..\ 

Once icost centers/ measures, and standards are identified! 
clear specifications for the processor mast be developed . ^ The 
guidelines must not be bu?;denscmie to the vendor yet meet OSFA's 
requlrc^sents . Vendors should be re<suired to. develop a management 
plan for implementing cost cohtrol procedures. 

OSFA should then evaluate the response received to .the cost 
control guidelines. This evaluation should ^address two questions x 
Can the vendor iiipiement a fiscal control system meeting bs^ 
requirements? ' And should the guidelines be modified* to 



I 



. 1,' 



acccBoraodate th^ veiiddr? In evaluating thSii^en^or ' s response # 

^the following .'factors Should be cbnsidered; 

• Direct resJ>onsiveness to the guidelines - 

^» Experience in pfer farming this function on o^her 
,^ , . '^^ contxacts ; 

; • K Innovative suggestions for improving cost control 

. Th^ ability of the vendor' s internal contract costing 

. ■ 'j' uyst^^ ;to -.«aeet,-th^ ' requirements, ■ ■ ' ■ 

, . , ^..^ ^ > ■ . ,^ - . . , . , , - ^ , . 

■ *> ■ , ■ 

Often/ contractors wit^v Federal contracts will have internal • 

■ . .. * . - ■ ■■- • ■ . • . ■ ■ . • • ■ - 

contract costing systems that will meet the guidelines. Thus, 
most vendors will not have to develop a new system to respond to 
OSFA requirements* OS FA might request copi^^ of reports of cur- 
rent." systems , although some vendors would be hesitant to |;elease. 

:the,;r6pqrtfi»."' , ' ; ■ ■,■„.*■' . /'^ : ■ ' • /..v- 
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laBVSLOp FISCAL C OIi miOL gYSTEM , 

\" One of the first tas)cs to be done should be to develop a 
.fiscal control system' This "wpuld be done jointly by €^FA and 
'.the^ contSi^or. . When evaluating the contractors* system* OSTK 
should notes , , „ . - , ' 

• The com|>rehen8ivenes8 o:^ the procedures ' - 
. . # ' Apparent defioienciea in the system 
J . # Contractors' suggestions that should be incorporaj 
"These items can be discussed by the contractor and ED 





hopefully, specific agreei^ent can be Obtained. • 

A bj^<S^et sl^uld be developed for each major cost cent e 

\ jr • . ' . ■ . . . ■ ■ ■ ^ ■ ■ ■ , , 

j/r ■ ' . „ ■ ■ ■ ■ ' ' ' ' . , , - 

wildch/wlXi beccmie by which performance will be 
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taeasuried . The budgets shauld be tied to the processing calendar 
along with t3^ actual production schedules r***-Ih this phas^, OSFA 
ail* the contractor shouia joint^Ly develop a budget for each cost 
center. The steps required to, accoaplish this , are: ' • 



• • Segregate fixed costs tvpm variable costs, which vary 

* in relationship to workload (e.g.,ni»nber of appli- 

."cahts)' • ■' ; ' ' ■ ', V 

# Develop a factor for relating variable 6osts to work- 
load**MOlime for budgetary purposes 

This process , can be done on a geheral levels since audit accur- 
acy is not^required,. It is important to recoofniae that very few 
costs are truly fixed/ an(|. variable costs usually do not vary in 
direct relationship/ to workload* OSPA axuJ the cphtractor "should 
retoeinber that the initial budgets represent a fiscal control 
standard to monitor spending ,• a %«>r3catole tolerance lev«l ceui 
serve as an effective early warning system to potential major 
cost overruns • Theifeforei the relationship between calendar and 
budgets should be thbiight through to produce^ exception reports " 
when a problem occurs, but -not »9 of ien J;hat the early warning 
reports ard\produced on a regular basis. Th^ final step in the 
process should to total the variable costs. Multiply theto by 
-the projected volume factor and adS them to the fixed 'cbsts to 
ensure tliat the projected budget" equals the bid cost. This step 
can also beetled to the production calendar.^ 

Early warning systetas can function only if they provide 'the 
information on a timely basis* ' Htowever, syetemis requiring weelcly 
r^orting will tetid to be inaccurate, inef fieTj^nt> and burdensome 
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to the contractor . Monthly reporting will probably not provide 
the information on a timely basii. Therefor^. "a biweekly (or 
semimonthly) system may be desirable. This choice must be made- 
by the; contractor so that it can be Retrofitted easi^ly into his 
existing accounting system. j • 

Three levels of reporti'ng are needed: 

• Detailed reports for each cpst center {td be teviewed 
/ by ...the; monitor) ' • • ■ ■ ,,. ' ' , 




• . Management simmiaries {to be submitted to the proiect 

; officer) . .^.^ .. , ■ '-^ .■ ^. . , ' 

exception 're;^rts (to be submitted to the project , 
• officer and the" on-site monitor) 

Keeping the volume of exception reports to a rainimum\ should be a 

.... 

goal of tooth OSPA and the contractor. Tlie effe<jtiv^ness of this 
tool lA l i be lost, if the vol time of repor ts is too great V In ai l 

^J ^f "^^^ will involve ad justy»ents,Y*P® with a new 

vendor. These ad jus tm^ta may include: , 

• Budget- or tolerance-level adjustments - * 

. • Requiring fewer exception reports for each cost center 
(especially on large contracts) 

Such adjustments should k»e agreed to by both OSFA and the' con-'. 



tractor 



^ Exception reports should include an explanation from the 
cohtractor detailing: ' 
_ . ^ ^.^a^. _ -^^Jfchfir . or no t-iAr Mets. a one'«>time onlj^ oeciirrence 

^ • ReasonCs) for the problem 

... - ,-Khen it .could occur, again. ^. , ^ ' 



Proposed budget mpd|.fi€atibns (i£ it can be offset) 
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Reports roust be delivered -^o both the on-site monitor and 
the'OSFA project officer on a timely basis to prevent surprises. ' 
We rec<Knmerid a one week timeframe. The on-site monitor should b^ 
contacted in person or by phone immediately when major (e.g. #25 
percent) cost overruns are detected— pripr to report issuance. 

DEVELOP CUIDSLINES FOR QSFA MON ITORING ' /' 

^ I H^a^.^ - l...^..-. -w nm I 1,1, mt%m ■miff xiMM^^^^^Mte^M^ i ■■ n mi , ^ 

Auditing of fisceLl control reports should be the responsi- 
bility ^of the OSFA on-site monitor. Detailed guideline^ for this 
jSrocess «iil be included in the next report. Basically, ^ey ^ 
inclitde; ■, 

'• ■ ■■■ ■ » •. ' ■ ■ . ' . ■ . . ■ ■■■■■■ ' • ■ ' : 

• Select ionv^f about 10 percent of the rfeports at tandom 
for revieW'.^ ' ■ 

Checking for mathematical accuracy ' V ; 

. Review of data for reasonableness (e.g., verifying ' 

that, all three programmers reported from a cost centet- 
• reariy are. there) 

The on*s:ite ajohitor should also respond to detailed questions 
raised 1^ the OSFA project Officer, following up with contractor 
personnel as required. 

The "role of the OSFA prc^jeict officer in this areia shot^ld 
basically be tliat' of "a liaison, responsible for: 

• Review of reports for reasonableness, requesting 
clarification as required 

, ■ ' . • ■ ' ' , . . \ , 

' • ^ -S^^^^^i^i^^^ J-h|03rm^ti^n to^ 

m4 R^contiiling reports to contract invoices ' ' i - 



J 
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Invoicje cecpilciiiat ion should proceed smoothly. (In the real 
yiarldt problems Will occur, but this is a theoretical discus- 
sion*^^) It ahould be a simple mathematical, process of adding two 
reports for a month and comp^ing the total to the invoice/ 
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t^.' ■ 'Chapter 4\ ' 

S SOFTWARE QUALITY ASSURANCE 

^AnT effec-tive software Quality Assurance (QA) system caft 
^ensure, Vithi ft reasonable limits, that all software meets- ED 
^standards and specifications 'and av^ds the countless expenses 
a«4 delays incurred when software problem^ ar^ discovered after 
the system is in the production stage. The problems that result 
i f rom an absence of software QA are obvious. Minor software 
: errorp can cause ma jor^ problems in the normal processing cycle . 
For exampl4^, minor software errors in pell processing can cause 
SARs to be distributed with false i^aforroation, possibly^ causing 
incorrect vawards. yifvn. error detected in the production stage 
usually requires temporary s)<utdown of the system {iossitiily during 
peak c^eratioii. When the s^tem is in production all riew\applica- 
tions must be reprocessed along with the backlog that accumulated 
while the system was shut down fca\c0ifrect ion, adding to prdduc- 
tion costs. Software oorrection^.^de quickly often c^hi^ot be 
tested as thor«Highly ?is they ^oijld be. This condition often 
leads to further compatational problems. 

This chapter ^utlines strategies which, if implemented, 
could result in an effective processor software OA. system in 
dSFA._ In order to avoid the_occurrence of ^^^^t^iar^^probler^^^ a 
"number of iju^lity Assuranoe/Quality Control techniques can be, 
utilised. Each strate^ has specific advantages ^ and diaadvan- 
tages, whiclt are discussed in detail in Appendijs B. The 
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following techniques are dOi|i«l(iet©di. 
♦\ Soft-war^ testing 
, • Software verif ipatioi? 
•• - Software valid^tiOB - 
' • . Internal cont rpl tdchaiques 
jij coniblnation, these t^chaiqies can lead to effective verifi- 



cation of software additions 



changes that c^n in5»rdve 



overall effectiveness of ^oftw^ire already in production. Si^nce 
time and resource's <fedictated 



Quality .Control jare' always 
f oc\^ on practical methods 
'designed to maxindsie the ^ualit^ the software. - 



limited, t^is discussion will 




QUALITY CO^sioL MP^ALItir ASSUHAKCE 

Quality Control in sof twkre tef ers, to tl|e^procedure|t f ol- 

4 owed by the contractor to enbure that, the software functions 

properly on a day-to-day ^j^s^, in4lu<sing the^ performance of test 

procedures for minor software modifications. Quality Control 

refers to procedures ^at should be f$>llowed by the contractor, 

described in the section on' infortaal testing . • ■ 

Quality As supancW* refers to tMe pjfocedsires^ used by OSPA ta 

verify t^at the specif icartions for software design and modifica-f 

t ion have been implemented correcic.ly> Since the contractor is . 

■•7 ^ r ' ■ 

almost solely responsible for all m0d£f icatibns to the system, 

the role of Quality Assurance is? to verify carefv^liy as 0Os- 

sible the correctness oi the entire system, using a coujbination ^ 

'of thfe applicable "techxiiques described in the following section^.; 
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SOFTWARE TfiSTI^i^! * ' \ , ' . \ " .1 

software testing is a^,,>^nctxba that should be perfon^ed by 
the ptodesaor. .Both informal and formal techniques are 1 
considered', below.' • .■. - ' ; '\ ,-. 

<:omputeir^Based tes'tjng {informal) "'-I ' ,,' | | ,. '„ 

' Informal ' cbmputer-bj^sed testing can be perf gsriied by ,the 

contractqr while modifying .the systefa. So formal .report^ • 

■ . ": '''',< ■ -...^ ' ■ ^ . . ,\ ■ ' ■ ' ' > ■' • ■ ' ■ 

•approval f'rom^EDU'i nepessary. Thie informal testing should be 

■ ■ - ■ . . - ■ \ •. ■. : \, , ■ )'"■■■. . 

part of the ttormal activities d*f pfbgranuiiers and systems J 

■ • ■■■ ■■■■■ ■ ■ ■ . / ^ ' - ' ■ ■ ■ 

analysts. Four .types of tests should be performed as paitt of th* 

ongoing modification or docudientation process » Often sevjeral oi 

these tests can be performed simultaneously. ^ \ 

. , ' ■ ' . ; ' .■ •■. • ' ' , ■ ' . .i 

• ^ Unit Testing— ^Unit tests' are performed cm all i|ndivid- 
;y ual mO(^ies cs»npxled separately, it is necessajry tc^ 

■:^ check, that each module functions according to the 

specifications from which it was developed. At #ac| 
step in the design and it^leraentation process, the 
„ correctnees of the logical structure shouldL,be qpn-/. 
\ firjaed bjr the coders and pro'grammers . - ' 

..'n'- Subsystem Testing — After con^ let ing the "unit, tfest' khe 

program module i^oul^ be tested as part of the test for 
i* the su^sya^^em to which it belongs, subsystem t^sts 
V , evaluaie the performance of a group of pro^t^m r^odules 
. ■ tha^ , normally 'execute* togegther as, a. unit wa-thin |tiie- , •;. , 
■ ^ overall system, problems with individual moduleti are 

not always identified until the modules are requijred to., 
. function within the frjimeworlc of the subsystem* ' | J 

' • Integration Teflting* ^integration testing chedts. 't^& way 
in which aH or some of the system m^aules are cbn- 
hected. Specifically* the communication between t^ 

- _ -system aradules ' is :! j^vdewed . .to., asaure . thitt the , fiubtyjitera. . 

can receive and paes data betiteen individual ^dulee . 
Siiftilarlty or compatibility of record a^d file formats 
used the modules is one "possible source of problems 
■ . identified i?y this, fnrocess. ; ' . : '>--.; '■■ \' 

' e Component^ Testing — Component, testing confirms that'' a ^ 
V functional unit within the system per fbrms correctly. ' 
Functioiml units or compor^nts can consist of \par|i ^jr^: . 



all of a subsystem or . can cross, siubs;fsteiBSi Xn a 6pni- 
■ _ ■ : pons nt^ test tbe functional uni-£' is tested by vejrifying 
, that all the parts of the systero thart. deal with the ^ 
particular functional unit can be eatecuted t-o product 
. the correct , results For exampl*! in the pe 11 ^proces- 
sor son^ functional /units in the corrections subsystem 
llare th^. processing ^bf student requests for duplicate 
SAR's and tfie processing of address correction 

■ ■ .requests. 

Computer-'Based Testing ( Forpial )' 

Foraial coraputer-based/ testing refers to th^ thorpiigh ^est- 
ine that, follow^ custoraer-^approved test plans. The tests are 
designed to confirm that all system software n^ts the specifics- 
tions exactly. Formal testing is usually performed vrtien the 
system is being evaluate^ but before its initial production use. 
The following aie four different types of formal- tests: '* 

• Subsys-teip integration Yesting ^^Thia is .Similar in pur- * 
/ ^pose to int^ration testing at thfe informal testing 

Stages. During formal testing a imich wio^e conjplete set 
/ , of tefts should be specified to ensure that the connec- 
tions between ail program mPdules are tested- a^^ 
* reported on, Sometirofes the output- of the individual'- 

. programs and subsystems will be sufficient evidence of 
correct operation, in othW c«.ses additional reports 

■ ■■ ..liiust be e^afelished* ; v ■ ' ; ■ ' . 

• PrototypS Testing^ -^Prototype testing is used to check 

■ ' : .V . ; ,■. .the. Correctness of ' the ".basic , design' of; program modules 
■■■■^.•aubsystems.' tests -^re^perfor'nied'- ori /a system which 
is a precursor to the final system and may not include 
. ; all the 'features the ^inal system includes. :prototy|>e 

testing can often give early insight into design prob- 
lems which can then be remedied early, making for less 
rework in the final sy'stem. 

• System Testing — ^System testing is done from detailed 
. ~ cUstoitier oiJtpu^ quislified per- 
sonnel.^ All initial input and final output must be 

' correlated to ^ill intermediate input and output. The 
system test should confirm that the system specifica- 
tions have b&eri followed pre<tiseiy and that the sy&€eiB 
is completely operational. If not, the system test 
should identify the program units or subsystems wherj^ 

■ . error^s have •occurred, ',■ • 



V' r • ■ Acceptance Testing — Acceptance testing is similar to 
■■' : » system teisting »but: it deitermines whetTn€»r' or not tfte 

system can be accepted knd put into. productioQ. Each 

• subsystem is evaluated separately and p&rts of the ' '* 
system may b^e accepf^ed" while others require further 
mod^ification or correct iotts* When there are no bugs or 

* 'only \?ery minor bugs* the system (or -subsystem) can be - 
. acc9,pted f6r. production. It is not desirable to accept 

^ -V . a syst^i'With erarors pf any type remaining/ but due tt^ 
., tinje presstire" it is sometimes possible, to accept pro- 
' ; grams with errors such as iraptoperly forma tt6d outputs 
». ' ."Extort oi this hature^ do ^not .affect .the prcyper func- 

tioning of the programs. ' ^ 

Manualr-Based, Testing- l ■. , ■ xy '■ ■ 

Three; Cypes 'of manual software 'testing .^re^seful .in the. ' 

' evftlua;tion of ttie design and completeness of system sof twatie . . . 

—Mamal testing ttricoyers -ert-or^ before the system is in pro- 

duction and is oiije of ' the ohly ,ways to discover <:ertain types of 
errors* .Manual chepks- can improve the most basic unit of tbe • 
system, tbe individual prpgramJ^^^^^^^^^^ program level 

reflects, on the quality «^nd ' efficiency o£ the whole systejca, ^ 
Manual quality cdhtrol is one of the leffsV. costly meth'diis of 
software testir^ W3 i«.also one of methods most seldom ufied. 
MMch more 'eraj>hasis« s^hould ]^^^ on the following manual tech" 

Design EvaluaCi^n'~ ^Oesign' evaluation refers to the 
. careful exeuain^tToh of the system" And Subsystem design 

by studying specificatlotis documentf^.^ Funotional/i, 
. ' requirements r system specif icStion^, and program speci- 
fications are reviewed in search df design fiavrs, inef> 
.ficient assign, or overall inconsistencies in systeim 

..J.:!,...:..:.:. i,,,..- J.„._:-.:_j..i;.i.:L:li._l,.^ 

P^pduQt 'E\^aluatipn- '-?rbdUct evaluatiori refers to the 
; . review .of the actual j,prc>^ram codfe and also syst^ and 
*," " iprqgram dopumei?tat4o».. ' Th¥ loai9'«^^^eff,^'^ie.hcy. of the 

' ' oode ate ,«X2UBined by ^,aa cxpetieni^fd prograara^r o^ 
" t^ian "the auth&r of tl!^ prog rari. ,<-This evalukti6rt should 

.•■ *>•■' ' •)/' • • - ' . > . ■ 'J' ■- ■■. ■ ■; 

■ • ^ ■ . . ■ /, . ■ . M . / , ■ ■ ' . , ■ . , . 



be performed before th« ccwvputer-basect testing £)f the • 
•system. • Problems founiS and corrected at ,thi©r-a,tage ' • 
will, simplify the later ey»tem testing stag^^n^ 
iin|>rove the quality .of all the programs and hence of. 
^he overall systisia. The accuracy o£ the software doc a-/ 
mentation can be verified at the sabe time as the code 
■is reviewed, ' ■ ; •• /- - 

Structared l^rogramnving Technology- "~Structured program-- 
■ming» also Itriown as to^-down prpgrantming, can be used ; 
to improve tbe quality of the code/ to &ta,'ndardi2ie 
internal program organization, and to ma]ce' program code 
more easily readable to other programmers.. Programs 
flowcharted and written using structured programming ^ 
. techniques contain fewer ^ errors to begin wi,th and aire 
much eai&ier to debug because they are easy to read, 
even in the absence of program doc cementation. Another 
structured prograimfving technique is the walX-through— 
a set of. techniques performed at .different times in . the 
software development stage. The_ waiX-through, executed 
by a group of technical members of a, pro jedt team, v 
reviews the work of all eu^loyees. Program specifica- 
tions, test preparations, code, and. output are all 
reviewed for correctness and compliahce-wxtih ^stab- 
■'aished vdtandards. ■ ■ 



SOFTmRS VERIFICATION AND VALIDATION 

Software ver if icaticm and validation techniques can be used 

by OSFA for quality assurance of the processor function. The 

■ ' ■■ ^ ■ •-' , ' . ■ * " '■■ • ■ ' . , 

following techni<iues' couia.be Integrate into proCe« 

-contracts r,. , ■ ' ' ■ . , , ' ♦ - 

I . • ^ ■ - ■ «' ■ ' ' * , , ■ ' ' '■■ * 

' Verifio4ti.dn , ' ■ ,4.'-:. /■ " ' .r 

Software ' veri f ication is the process ' ^f . detsrmihing , Whether 
the re^ultr' of executing,, system poftwar^ in a test- /"environment.,' 
agree with th^ briTgihal ^'cificatibns. Verificatidn is con- 

" cer nsd""wi t^ T tHe 'cor rect.ne«8 ' the', ' I'bg'ic "of "■'the' ' aoflware--: 
that is/ making sure the boftw&re -satisf iSs- the functic^nai 
requirements ..definad ' in/the ssp'^if icatians . The'; following Six 
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techniques used .to verify software are normally used in 

conjunction witii each other and/ or other techniques. 

• :! • . Drivera' " Driver programs are written to perform unit 

tiesting and some integration testing in the implementa- 
tion stages of /software development. They are super- 
fluOuisN;ode that will' not become part of the system but 
■ )■ are needed to test program modules. Driver programs 

are c^ften used to generate test data and to drive, a 
grou;^ of other programs. 



• Test Data Base- '-A test data base is a . col lection of 
data f that closely matches the actual data that will be - 

used in the production cycle <5f the software system. , 
The test data should contain records and files that are 
■ repre^sentative of the variety present in the' actual- _ . ■ >■..■- 

. ; ' data/ Although the quantity is usually considerably >. 

-less.' It is also important to create records, that con- 
^ ■ tain invalid data to ensure that the software properly 

' edits ( flags, or rejects the data systematically. "'"'^^ 

• Desig]^ Verification- — Design verification refers td the J 
examination of software specifications for the purpose «; 
of finding design J errors before the software is coded. 

''r/:-:'^ contractor has created the preliminary design 

for the program modules, according to cWtomer specifi- 
cations-, the customer reviews the softwa»fe design. The , ? 
custoni^r is presented with detailed technical descrip-:: * 

' > tions and flow^arts of what the software modules will ' . 

; * be like wheh con^leted. This avoids the creation of '^'l^ 

useless software that would need to be modified at a ' 
higher i cost later in the develppment stage. , 

' • E X ecu^fi^n A n^ ly s i s — ^^Execution analysis refers to the 
, automated monitoring of cc»tput^r-based software testing; 
^ and the collection of data froia this testing. The data 
are t|ien analyajed manually and tmed to predict the dur- 
ation and cost of testing and also the quality of the 
software product. 

• Automated Network or Path Analysis— Thi». is a technique 
that analyzes the program module source code to deter- 
mine the minimum set of paths through the code that 

„ will nmlce use of all logical brajnches of a pi"ogram _ _ 

^ ""'m'odule.' •"Thieytedhhi'que"can''''be~us^^ ---- 

implementation- stage of software development to opt i- . \ 
mise code efficiency. 
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. . Statis-tlcal Prediotion «— "in this technique errors are 
inserted into the progr^un mod«le. code and the program 
is tested ^ntil all errors are found. A confidence 
, ' factor is then ^ora^uted which indicates the effect^-ve- 
ness of the pro^rainming and verification process . , 

Validation ; , ' 

Software validation is the process of determining whether 

executing th^ system in the production envirbnmeht causes any 

problems. Ihie system in this case refers not only to the soft- 

ware modules but also to the hardv^are, user procedures, and 

persohnel. ' Validation is concerned with confirming that the .4 

specifications and produc-^ion environment information (used 

i][iitially in the designing of the system) ar6 complete and 

■■, ; ■'■ , . ■ '\ . 

correct. Following are three commonly used software v£»lidation 

.■. :. , • '., ■ ' ■ ■ ■ i.:. , ■ i . - - ■ ■ ; '. . 

techniquest ; | ■ ■ • ' ' ■ ; 'V 

• Functional Testing — Functional tests are designed "for 
' a very speci^f ic functional capability <>i a program 

module or software systeni. The emphasis of the func- 

, tional test is on the specification rather than the 
'overall prog reua module. It verifies that the user 
rej^uirements have been programmed correctly ► The test*^ 
also can be usati as a verification tedhnique. 



I- 




P • Depigri Validation — ^Design^^^validation refers/t^ the 

procedure of designing a software systAffl for tinding 
prograutt 'errors * This t^ehMque is siiftilar to the , 
; design verification technitj^ue, but it occurs earlier in 
the software development stage. Design validation is 
basically a review of system software plans which are 
analyzed raanuai;Ly to maXe. project iotas for upper man- 
agement -and custaners on project status, existing » 
problems, and Whether or not schedules ^are being met. 

- - Matrix Analyses and Problem Statetnent^ i^^^ " ' 

technique makes of a forms-oriented language or a 
formal syntax language to eamnunicate tJ>e needs of the 
user to the/ a;nalystSi The us^ of this technique can 
• faciiitatfe th«: detection and elirainatioci; of logic 
errors by the u«er because the e^nalyst and the user 
wo t)t close ly in evaluating system requirements. 



internal Gon-trol Techniques * j*. 

Internal controls ar^ statistics and edits written into the, 

program modules and designed to perform internal checxs^n the 

performance of the. program module, internal controls produce 

data to be used iii perforgiance monitoring of the program. Six . " 

internal control t^JftujOjOpes are described here , . 

• . Limit Checks— Limit checks in programs cart be used to 
idientify f^taLand combinations of data likely to be • 
> inval-id. Thejprograms use a< series of software 

instructions that compare certain conditions and refer 
to a table that contains high and lov^limits for para- 
meters, ^en errors are detected special reports are 
created that report the error conditions to management. 
Approf^riate action nuist be taken to remedy the prob- 
lems. 



Record gpunts— 'The internal counting of records and the 
review of the ciauntjs is e#sentiai to keeping track of jj^ 
data withirt a system and verifying that the programs 
are pa#slM^# sorting or updating files properly. 
Counts *aref normally performed at least twic;6 within a 
program-- once wl^en the record first enters the progreuu 
and again at an appropriate point/ usually immediately ^ 
before it is passed to another file or program. 

Batch Tatals and Counts — When a specified number of 
records ^e^processed together as a batch, as they are 
in Pell ^raht processing/" it is necessary to keep track 
of these balches^ as they , flow throt^h .the system, tfeis 
■ * mJ^^- ^ calculating a batch total on a datay field 

■t f^ommon to all records in the batch. The batch total, 
batch ccmnt« and batch number, are then reconciled at 
apprc^riate points in system p^'ocessing- Any discrep- 
ancies must be researched and corrected. 4*^. 

• control Totals — Control to4:a Is are Statistics computed 
on significant data elemente before they are processed 
by the system. -Totals on the same data elements are 
calculated again during or at the completion of proces- 
■ sing by the software system. Control totals" give an 

r - -"'estimate of the i&gnitude of the Bof tware -errors in the - 
system* Record counts provide similar information but 
are concerned only with th# voiume of errors? control 
totals are concerned, with the value of errors 



Hash Total g' " — ^Hash totals are calculations computed on 
xnsignxfxcant data such as Social Security nuntoer or 
serialized document number* Hash totals .are calculated 
as the data • enter conaputer system processing »nd during 
later system processing. Differences between these; 
hash ^totals indicate missing records and sometimes can 
be .used*tp identify .the specific documents that are 
missing. Y 



Filfe LabelB- --^Internal labels can be placed on magrietlc 
tapes to identify data recordis contained on that tape. 
Header labels become the first record oii the tape and 
can;be verified by program sOftwar^ so that wrong tapes 
^ are not- read by the -^program, ^ The program can stop pro- 
% cessing immediately ^nd alert the production staff to 
the erroy. This will save processing time and expense. 
Trailer labels often contain record counts and ^control 
totals^ as described previously'r ^ 



ADDITIONAL" CONSIDERATIONS 



T; 



The following are additional issues that should be con** 



X 



sider^d try- OSPA when developing OA and QC. guideline^ for pro- 



cessor contracts. 



Privacy and Security— -This i s a complex problem Whj&i\ 
large volwoes of information are being processed 
through a production 's6ft.Ware system. Basic precau- 
tions should be taken at each step where the data are 
handled ^ productlopt staff , ^Personnel must, be 
informed of ' the proper raetBods for treatifig confiden- 
tial djeita as well as their responsibilities under the 
Privacy Act of 1974. After the data are keyed onto 
tap^N^r disk they are not easily accessfed tmiiftentibn'- 
ally ^o security becomes > matteir of limitit^g access to 
the ^[ata and software to only those authorized to ^ 
handle it. 
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. CjOmpUFter Systems Costs — iThere are numerous ways to 
Vinimize the costs o^ operating a large software sys- 
tem* Many of the previously disic^ssed QA procedures 

•cm tihe program module unit up to the entirs system. 
Although there is an initial personnel cost in imple*" 
raettting these OA techniques, the cost of operating the 
'^■system on a day--to-day bSSis 'can-'b® .reduced drastic- 
ally, ti is 'important to ke^p 'central , process i«^ unit 
(CPU) - time and size to a . minimum. 
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, Program Change Controls- — Ml ?3rocjram module aoarge 

codas Should be protected against unauthorized changes. 
The usa of a secure source library -system, like 
,PANVAI<]BT» can help control usage of a source code. The 
object code should also be protected, but due to its' 
nature* it ^s not usual iy as easily modi f£ed. 

• ' Documentation— Coroplete documentation for a l^rge soft- 

ware system »«:h as the Pell Grant system should con- 
sist of at least a maintenance manual, users' , guide, 
V. . and systems overview 4ocaraentation for nontechnical 
. personnel at IJJ. , ',1 ^ ,^ ■,. 

" • production Control- >>A detailed plan >M»t be written 

that is specific enough to be followed routinely by the 
production staff bCit flexible enough to account for the 
daily fluctuations of quantities and types of- data. \ 
Each ^member of the production team should be assigned 
^ specific types of duties and authorized to perform only 

_i these, duties.' When problems of ^ny type occur, ttie v 

method and the person responsible must be clearly 
./^ defined. - ' ■ ■ ' -'• " ■ : ' ^ 

• Data Cgftgr^ -wThe handling and storing of data should 
be out lined tin the production plan. Data must be seri- 
aliaed and filed in an organised and sepufe manner. 

> Proceduras Jbr. transferring data must bje set up, and 
■ > all data pi^essed by the system should be cleairly 
t^^^' fin^l destination so that qper a tors and 

production staff do not misplace any data. Careful 
daily documentation of processing will guarantee that 
all new data units (tape or disk) will be easily accea- 
Siblo when the;^ are required in future proceasing.. 

'■ ' y ' ■ ■ ^ .' ■ '■' [ 

• Magnetic Media C6ntrol~ ~£n addition to the careful 
organization and fixing of tapes, previously described, 

^ they require addition 1 car.e. Tapes must be protected 
^ against environmental hazards. Backups should be kept 
iot all System tapes. In order to reduce the volume of 
tapes procedures, should be established for disposing of 
those no longer ^heeded. 

' . ■ , ' ■ . 

CO^CI^^ION 

■ . "Software '<iua 

» ■ - ' ^ . .. . . * , ■ . , 

important function in an overall quality control system for the 
student aid processor function, jn the. final chapter the tech- 
niques ouilined jiere are applied to the Pell processor. 



f 
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PBtODUCTmTY 

the processing fiinction Is cpo^oae^ pf several production 

' ■ ' , ... ^ . ' 

■ ( ■ ■ . . ■ , ■ ' • 

dte|>s« some of Which, involve manual activities « Pairticulaf ^ 

attjent ion should be ^id to manual 4<^tivities because s 

e They are usually the slowest activities in. temw of , 
t ' processingi.. ■ .< . .■:\ Vk.' 

' • Labor costs are usually a very high percentage of 

t;:otal costs ^ so -for a given processing system these 
costs should be minimized. • » 

' ■ . ■ , ' . . 

• In prc^esses with unevennes^ in processing <i.e«, 

peaks- and valleys)* the labor function usually becomes 
most sensitive to changes in level of processing. 

• . .' ' • ' ' ' '■ ■ 

OSFA monitoring of manual activities for efficiency should 

not focus on personnel efficiency. Instead, it should focus on 
how personnel are fit into the production steps . These steps 
were outlined in a general way in Chapter 2-^- that is, the activi- 
ties should be scrutinized for how well they use gersonnel in 
processing rather than for howyf^oductive personnel are . The 
most efficient production process^ is the one that most effi- 
cient.ly combii^s peraonnetL and mechanized devices. Top imich 
labor per unit of machine resuX^s in excess capacity, and labor • * 
costs Vill be too high. Too little labor will slow down pro- 
cessing, and accuracy pr oik ems may occur. 
~- ---The "inost- direct way ^^^-^^ 

process is to be^in by looking at the different prpdufction steps 
and the laanual aetivitiea within thesa. pieae steps should coin-r 
cide with the cost centers identified in the proces^. At this 
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data on the number > cost^ and time of pfersoi^el. and ^^^^^ 

■ ■' ' ^ ■■ •■■-■■V, .\y .' ■' . / ■• 

: cQ3t;3,. caii- .be gathered ar^ qompaired., r ^ . ■ ■ 

STAGES IK.MQt?ITOSING PRODUCTIViTY ^^^^^ ^ ' \ ^ 

.. — ' — — ' T- ■ ' -^^^ 

.' i There are two stages ia monitoring productivity. ''Th^' ^irst ..^- ; 

■ ■ ■ - ■ ' ■ ' , / \ . ' , - -^..y- • •>■ '"sX,.-^"'-"'',^ 

Stage la establishing efficient processes for laanaaX acti^itiisd. t^^^^^^;^^^^^^^^^ " 

^ ■ '^^ * ■ ' ■ ' ■ . ■ '. ■ ' ' ■ ' - /■••r-"''^'' 

This can he taken as a I given, or it can be varied if it is deter^ 

(nined to be ii»f f icient,* The, second atagfe is the ongoing moni> 

toring of resource requirements and the mix of personnel and 

mechanical devices over the course of changes in th« parameters^ 

of : the process, including the level of incoming doauraeiEits. This 

two-stage approach tLs analogous to establishing measures and 

standards. .y-^-- 

The estabj^ishment of standards i^ust rely oh the ;iudgment pt 

OSPA iiianagers, - experience* and the ability of the contractor to 

be flexible. With a lack of information on histoirical trends on 

productivity, a method for assessing maximum efficiency must be 

• - ■ ; ■ : ■ . ^ . - ,' ■ ■ ,y ' :.■ ' " ' ' ■:■ ■ ' 

devised* ' Since is^echanical devices are labor- saving, they should / 

be used as a standard of performance. The number of these 

devices that can be used and the number of p^c^l^. ^ho can use one 

device Should be calculated. Observing working patterns of idle^ 

..^ , . .^^^ 

give a rough idea of how efficiently personnel are being uaed. 

l^hangihg' the .ratio of personnel" to machines "ahd seeing what ' ' "~, 

' ' • ■ ... . • . ' ' " . ' ■ ■ , ' ' . 1 ^ 

effect this has on processing time should give an idea of the ! 

. ■ ' / ■ , ■• ■ : . ... ^■ . ^ 

optiioal mixture of personnel and machined, in activities where 

^ 52 



ere are no machines, observing processing or activity time |pid 
vairying the nuiolsor of personnel should giv6 an idea again of . how 
ttte aotivity may work more efficiently. 

/■. . ; r ■ ■ J > 

In this initiski step, <^ta should be collected on. the nuiober 
ar^d type of people, how much they worK, the number of^ pechanicai 

devices available, document volume, and turnaround time. Multi- 

)' ■,.■ ' • ■ , ■ ■ ■ . * ' 

yairiate Statistics of ratios per unit' o£ volxxtm or ratios by 
^turnaro^nd^tinie should be cc^putedi. These, tbeh, becoine tHe 
standards to Which subsequent £K^sures are coii^ared> Th^y hold 
cbtistant variable volumes and turnaround times* 

^ The second stagfe of the nwDnitpring procedure is merely 
collecting the same Kinds^bf data and computing the appropriate 

statistics « X£ the relationship^ betweer^ labor amount and vollime 

. * ; ' ^ ••■ .- ^ ^ ■ • ■ ■ ■ . : ■■ ■ - .. . ^ . ■ ■ . , ■ ■ . ■ ■ ■ - ■ ,• ;.]■ ■ 

^nd ta^^round time are stdtblSt then %b volume grow&V it viil be i 
possible to predict how many additional personne 
required and what their cost should be. TOtai contract cost for 
a small time peiribd will be directly estimable* AXso^ if the 
statistics begin to vary signifiCahtlyV it should, be possible for 

maQagement to pinpoint approxi^tely. where the bottlenecl^s are 

, ■ ■ ' ■ ■ . ^ ■ ■, , ' 

and how to solve them. i 

■ , <^ ■ ■' ♦ ■ ■ :• 

This kind of ^nitoring is actually -easier than It jsomvas 
because the measures are baaed upon the assumption of ait^able 
relationships rather than ecoi^^ies or disec(>nomies of ^c^le in 
the processing function. That is'/ if fixed costs must be changed 
drastically at sc^iie threshdld level of document volume, or i^ the 
training facilities Vpr personnel are hot kegt at the same pace ■ 
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as the need for a particular type of feirsonnel . then the "statis- 
tics will -not be -acburatev . . ■• .i.:, • , 

TASK SCHSDO;.ING AND COST HANAGSMENT I ' / ' ' ; ' 

For extrepely large processor contracts , OSFA should con- 
slder estahlishing, jointly with the contractor, a task iachedul- 
ing and ittdhitoring system that would enable OS FA project oKanitors 

• Monitor jperformance on a;previously established , „ 
,. ,: performance schedule ^ X ' , _ 

■ • ' - ■ . ■ -"* , - ., . ■ ■/■ , '. ., ■ ■ 

• Estimate the cost savings of changing performance 

\ s standards in various steps in, the productibn process 

. . . ■ ■ ■ ■ • ■ • ■ ■ •[ ■'■ ■ . ■■ ■■ 

, Est^lish a* reliable mechanisar for estimating staff 

requirements for each step in the production process 

,r • / Estimate the cosjts* of major delays in the j>roduction 
ache^le, such ^s congressional delays in Reconciling 
, the program budget or final^ Family contribution 
Schedule 4 



■ ' Currently, a number ;0§ automated systems could/ be used for 
this purpose, -in partiq^ilar; the Deparjtmeet of Dejcense uses 
these strategies for tl^; management - of lacge defe^e projects . 



If OSFA were to use this 
avetlabi^, including;, 



/strategy, a variety of options would be 



Requiring the processor contractor to establish such a 
system and to UseXj. ^ot production control » cost 
control, and piioductivity management , 

Establishing a fcpst control scheduling system in OSFA 
that could be used to monitor the OSPA contract - 

issuing a separate contract for the purpose of cost 
control and scheduling of large processor vendor 
contracts iA OSWi 
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The aalection of a strate,^y ^or developing a task a^heaul- 
ing and cost control aysteio; deWnda on the goals of the 
Department. ' '^If the paroceasor cpntractor jjiaintains such a aya- 



tem, the fifat ia the moat viable 6ption. Since oSPA does. not 
currently have experience with "auch a .ayateta, the .feaeil>ility of 



,e second 



two options ia limited. 



,Th^ uaie of "an automated taaTc 'acheduling and coat monitdring 



aystem has f the potential to improve production efficiency for. 
mahual activiti.ea.» If ^ the two-stage prbcess described "above ye re 
used t<3» develop productivity measures 'and .standards thesa 
requirements could be.^ entered into the systert ajvi monitored as. 
part of th4' overall system. In fact/'alJL aspects of quality c 
■ trol and procesaor contract raanag^m^^n^, considered here cquld . be 

through a cost contciol system* *This issue 
Will be ieonfeiideijed in mo^e depth in th^ se^ocd raport , pn*. 



pjroceaaor *C 




jOC REPORTING\ MOfiHTORIMG, AND CORRECTIVE ACTION . .. 

The nrevtous* chapters identify areas in the .processing \ 
function vtiere quaiitj?<?3tmtroi standards and raeafimres could be 
estahlisSed. This chapter considers the wiys in vfhicti the pro- 
ceesor function should be taonitored and how the ini£orTnation '\ 

" ■ ■■ • ■ . /• . ■ - ■ • • V 

gathered for qua! itjr control can be conveyed to tihe managers of 

. „ ^ ^ . / ■ ■ ' ■ '.^ , ^. \ 

the system. This chapter does not deal directly with the quali\ty. 

■„ . ■ ; , ■ ■ • ' ; ; , , ^ , ■ ■ • A ■ 

assurance role of ED and how it should pe^fora that role. This! 
topic will be considered in a second repbrt. Instead, this cha^~ 
ter considers trhe relationship between OS FA and the contractors i; 
for processor contracts. The three critical areas for monitoring 
QC components Of processor contracts are: . 

.eporting ' ■ :■ \- • '■'^...s-. ..' 

//' -.--. , : - ■ ' . ^ ■ ' ^ ' ■ . . . , \ ■ ; ■ ^' 

Monitoring . 

Corrective action ■ -.'^ ■ . • i 

Quaj^ty control-^ reporting is the way in 'which the data collected 1 
on/ tJt9 perfor^nce of the production process are conveyed to OS FA 

a;Hagers. The^data should be timely and accurately ' reflect the 
.ddhdition of the syst^a without overburdening the managers witrh a 

vmpbntain of useieis statistics. QC Monitoring is the method by 

■'/'.•''.' ' ■ ■ , • *''■■. 

which the production process is, evaluated # This involves who . 
'should do the monitor ihg and vi^at roles the contractor and. ED 
■ should pi ay in the monitoring . ' Corre 'may in " ~ 

Which problems ih the system can be..corriected jn<5st efficientiy. 




That is, how should a decision be laade on what corrective action 

■ . ' ■ ■ ■ ' i ^ . ' ' 

should be taHen, and who should make that '<3ecipion7 

* ■ • ' ' ' 

QUALITY CONTROL REPORTING 

If all aspects of processor <juallty control discussed in 
this report were io^lenientedf the productioti process would 
produce several reports based on data collected i^th^j||^£'eas 
describing the condition of the process* These repo|rt^i would ^ 
contcHin informatior^ on production cootttjil, fiscal, cjohtrqli 4 
software perfpirmancer and productivity* There woulii |be ra great 
deal of inf ortnfiition^-probably too mu^ be reviewep by 'the - 



mana^er^ of the system, the go^yjf^of tlte reporting system should 



be two-foldr 



1^1 



• The. reports should contain only the most crubial 
^ information on ^ routine basis needed by managers to^^ 

r effectively monitor the system* - 

• Various levels of reports contaiAing unique informa- 
' tion (or -just aggregated informatiori) should | be geared 

toward different- levels of tBafiageraent* 

For contracts subject to intensive and routine quality * 
control specificatioB^r the types of reports should be I s^jecified 
before the contiract inipleraentat ion stage. This reportf identifies 
specific areas in tlie processing funct ion^ that should pe mbni.-; 
tored- In addition, given the unpredictability of> th(^ policy 
process, additional information may be required.. Hov#ever, OSFA 
managers should decide which of the areas ident-ifiecf afe most 
crucial for nianagirig purposes. Information relateW to" these 
areas should be reported routinely to OSFA, -pjrojectf mohitors 
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' ."bther types of Infomation can 'be included 1/h the QC system but 
' reported on ohly v4i^n CX: standards are not met. • 
^ OSFA raanagemerMf should decide \^he timeliness of the 

and how often they shoiiXd be reported* The lag between the pro- 



.■y - 



daction^of data ap]^ the- generated , report should be rainlttiized for 
crucial 'data.:, . i » » - .. 

' For most of the large processing contracta, numerous people 

at need^p ki^w the perforroance 0f the proceasi^ 

■ ■■■ - / ■■ ■ ■ ■■ ■ ■ . ■ . * ■ ■ ■■■■ . . " -■ . ' . ; / • ^. 

Different' groups have different information requirements. Some 

■ '■■ ■ . ,. ■' / ■■ - ,•' ' •■ '" . •> - , ■ ■ ' ■ ' " „ ■' '■ 

groii^a are related horiaontally, such as managers in contractA . „ 

vAio are , in \charge, of controlling costs of automated data process- 

ing* Other nmnager 8 are related vertically. / Top management ih 

^OSPA should he concerned only v«dth suimnary reports or speqial , 

^pr^Jbl,^s facing the processor function.; ,Soi5e groups are interesl 

ted 4,n the actual Operation of the processing atelps, While others - 

- are gohcerhed" with how the processiiig function ts performing in 1.. 
the cpni;.ext of the overall delivery of the program. The reports 

must tefl#ct the divergence of ,the groups anS their information , . 

♦ • , • / '■ ' , ■ • . ' , _ _ , ■ ■ ■ . ■ ' 

needs. In the development of - the processing function* the tpp*^ ' 
l0vel^nagers of the system, must identify v»*io should receive 
What types of^ reports containing Ois^t types of. irtfgrmation. - 

The lovier-ieN^ el managers who monitor relatively small, 
weXl-defihed parts of ;the processing function should receiv.e the * . 

- moat -detail^- information on ^hose -parts -routin^y*- -^^^ infor-!?'—^ 
raation may be simply specific costs, within a cost centeT--0r the - , 4 

• number of applica-tions 'rejected or corrected for '.a|I -the computer , 
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'0clits in the system.' GpiTng up t|"he managemant ladder, the reports 
should be ^emx^d tO;the'.^inajor. interests of each manager arKi the 
4a ta aggregated to ^tha appropriate level; For instance, a roana- 
ger . interested ■ in applicant flow in the Pell progrivk may wajjt to 
Icnow only the nuniber of applications in the previous week, month, 
and yearj-t"o-date as collared to last year at the same time. Such 
a report is simple^, hut-^^ gives the foanager everything he needa 
to Xnow at a^ glance'. . The reports should b^ cl^i^rehensive but not 
^ filled with extraneous inf6rraat ion. 

QUALITY dcyTgtOl. MOtaiTORING ' . ' . 

The processor and ED should both monitor the p^rocessing 
contracts. The roles differ in that the conticactdr should be 
more concerned with the quality control procedures in the dy stem* 
while ED Bliduld play, a quality assutaince role an4 evaluate the 
contractx>r* 3 procedure^. Monitoring should be done on several 
levals/ ranging from individuaijjlj^oduction/s^ 

wj^thin tliose steps to- the overall systami - this demands that th^ 
6ontractot monitor and eValt^a^e *th^ system in an .ongoing way. 

The Riost effective means for an ongoing evaluation of the- 
. system^ i-s to h^ve the contractor establish a quality control teail! 
indepe^mS.eni: of the production process>, This team should spend 

full time 'evaluating the system- and trouble-shooting areas of 

■ ■. - ■ '■■ ■ ' ■ ■ , - ^ ■ . , 

potential proble^.\ The 'size .of the on-sfite. team should var^ 
' depending upowf the size* and complexity, of the production process . 
The 'ieam should obsei^ve the prpces^s'^ivi try*to anticipate ^ 



probl^s in it. ^ /way to do this is to have the team receive 

'. . ■ . ■ ■ ■ ^ . ' ^' ■■■>•■'. • ■ 

its ovci aet of reports from the, system; iji great enough iSetail to 

.identify potential problem?. . In tarn^ the quality control' team 

.should submi-t reports to the higher lev«l of management of the '* 

■ ^ . , . ■ ■ ' ■ ' i ■ . , \ ^ ' . . ■ 

contractor and to the ED managers. -In addition^ this team should 

act as an intermediary bet^en the managers and thS contrag- 
tor*s managers of specific QC steps. • 

A problem apparent iil\ the ED monitorii^ of processing con- 
tracts, especially l^rge ones, is the J.ack of intimate involve- 
ment in the day-to-day operations. As wag alluded; to earlier #^^5 
managers often split their duties "between a nximbeJ of activities. 
As a result, they rely on the contractor and on reports for . , 
information regarding the operations. R>r large contracts, BD . 

. should inonitojr contracts po-site, especially the ;^c^ual worTc 

site i9 more thar^ 50 miles frason Washington. The responsibility 
of the on-'eit^ nraYiitor should be to overSee the operations in a 
general sense and. to toake the immediate decisions necessary from 

y^ED. In ^addition, the on-site monitor should be the key coaKojmi- 
cator' of in fozTTiat ion between the contractor and OSFA and the 
fir^ point of contact for each" sider 

Although OSFA? should be concerneci about the -efficiency and ^ 

^effectiveness of the ^processing function, it should not be 

closely involved in' its operation^ Therefore, OSPA personnel 

•* ■ ' ' ■ • ' ' ^ ■ 

should not deal directly with the -quality -control aspects of the ^ 

contract. Instead the contractor should be the responsible for 

< ■ ■ ,' ' 

leeing that the operation runs smoothly. It is' more appropriat;e 



for OS|*A personnel to eva,iuate; the qiialsS' contjrol i^rocedures 
and determine how they can improve opei^ationa . The reports .OSFA 
receives ' should re&ect the Jsinds of quality control procedures 
the contractor has put in ^laca. ^ When the reports show a prob^ 
lem, ED should step in and inform the, contifactoi; that the bpera- 
t^i^ n|eda ! to he changed and then closely watch to ensure .its 
successful alteration. 

QUALITY CONTROL CORRECTIVE , ■ , — : / 

' A mechanism for implementing change in the processing tunc- 
tion should be established, and yhen a change is warranted for 
some reason/ corrective action should be taken* ^B^h the con- 
tractor and ^SFA should be told of the changes and'^their irnpli- 
eations. The ^^ernal quality control teata should b^ abte to ; 
ideniify needed changes and deteriaine , with the contractor how to 
make those changes./ The proposed method for monitoring th| pro- 
cess shoMd aid the, change procedure ( This information can be 
passed along to the on-site*" monitor who can evaluate^ the proposal 
and give some indication of ED acceptance. This can then be - - 
passed to the behtral office of ED which should make an informed 
decision, given the information of the- contractor and the assess- 

ment of /the monitor. , 

: If OSFA wishes to .initiate corrective actions in the pro- 
i^ssor function It can convey its information to the on-site 
raoHtor who ^cah" eva terms of the impact on the syatfem 

and then gather the contractor's reaction to the proposed change. 



The^oniipr can then aet^ as an interned iar^ \ii6 knows the operar- 
-fciqn M^l and yet represents the interests of ED. 



rreement on/ 



how th^ change should take place arid \Aat the' fiscal impact . 



should corae quickly because the lines of caramon icat ion are open 



\ 




f 



0 ^ 



1. 



,1^ 




1 



V j< .5' 



PRQCtsSoR QWALIT^ COtftitOL IN THE PfiLL GRAJl^ , 



l«.uer o^ equality •o.htr^^^ th. processor function for 



tfhe pftll. /Sijatttr Prograia ate coiiaider^ eX|>lictly in' this '^ihapter. 



5^ch,>ias two sec.tionss 



3 \i 



m-.:'": The^ dev&lopaQrit pf^ i^tandards \for the 

-;.;V,:. ;P^ii ;procesaQr \ System > - " 

•■ ^./;.^.Th^i 'qse^of ^a 'Bsutiplfe^fibrv^^ the vendor' ',; 

pr<k?essot:-'furiction,!^- . ^ ■ 

■ • ; Whiia ' iROsit. of pha dtata-., fteeded •: to " rbo-^itc^r ^'tl?.© - .systep ■ , ■ ' .-^ 

effectively shdjyild be derived frqra th*^ universe bf applications, 

some can be oJSsja^ned froia; a vS^un • A: samplie, can 

^be u'sed:_in t.w iskyss , ' : . • 

■ *, ^ As' a isip0 the fysifik in the production stage . 

-v/.^./ 'a- :.M^|iy\ tsea'$ures^;,ahd^ standards; referred to in . the 

foilo^idm section * use sa^ 

and so wrth to checl? 4 the accuracy of the fys tern. 



" As "a •; free-standing .sli&e "6f , the' .product ■ of ,' the ■syst^ . 
A sample of- appliciatiohs can/be usfd in a wide variety 
of ways beyond ihanageraeht purpose a » 



■ The. previops: chapters: '<?f' this-' report d^eipp- a ■frainev»forX ' 
for e^tablishifig a ey^em of measures and stWd a rds^^^^^ 
forraahce for different areas of student ai^ processing contracts. 
The fraiiiev^orit provides a Peil proi:fesaor 

contract ih/the^.^ll'o-ving^.'a.reas.*.* ^ ^. s" /. * 



• Production control , 

'y .m '■ Fiscal control . - . • ■ 

• Software quality assurance 
- ' • • ■ 'Product|.^ty , 

In addition* ttie overriding concerns of QC reporting, 
quality of data, and research underlie tlie discussion of each of 
the arisas. The guidelines established forsthera taXe into accoiint 
these concerns* If the correct measures and standards are 
established, and th6y are reported effectively, then the ^system 
should pr9duce accurate data in the most efficient method 
-possible*' 

• ■' • • , * ■ ■ ' ' ' " . 

The fraraeigprH proposes a methodology for developing raeaamres 

.... • - . ■ . y , "„ ■ • . ; • ,. . ^ ■ 

and standards for the generic^:steps of an^ student aid processing 

function. The purposes of this chapter are to identify the 

crucial areas for^.<x; within the lPe?ll application processing 

function and recaaraend g\^delihes to d©ve]|.op measures and 

standards of those areas of perforpsance* Appendices to this 

cljuapter contain sample reportx formats and laore specific guide!- 

lines th^t can bd used for the 1*^11 contract. 

Pyoduction Control \ ^ 

The Pell application process includes th^ forbduct ion steps ^ 

■■[.'' ' ■ ■ ' ■ J 

outlined in Chapter 2. Consequently, the pf5^ction process can 
be broken doVn into six st^pss* . 

Receipt C(!antrol — -The point where the/ initial applica- ^ 



-ti* 
MS 



on , "correction, ^61 ephone call., or data tape 'tro« an 
E site' comes into the "system" , / 



''I 



. • Cursory gdit^ -^Yhe point- at vhich the first stage of • 
editing is done oh the application or tape ^ 

^ \ * ^ Key saitry -^The point at viliich the data are, put into 

■■ '-^ the system ■ ■ ' ' ■■ ' ■ , i \ . ' ' 

• - Compute — -The autcsnatic processing system **iich 
. conducts edits on the data for consistency and also . 

ccmputes an SAI | 

• SAR Production— Whether not an SAI can, be ccMnput^, 
production of an SAR; Which is sent to the applicant 

: . ■ ■ , • ■ ' ' ■ ■ ■■■■■■■ . ■ ■ ■ 

• * Corrections Processing- *"Where corrections to data are 
* made in order to charage the data on the SA^ 

^ The objectives of a production control system are to: 

i • aisuxfe that applications move frcan one production' step 
^ ^ . to tt|e next in the least; amount of time v*tthout 
* ^ sacri^ficing accuracy 

■ ■ ■ ■.{. ^ ' .■■ . ■ ' . ■ ■ ■ ,, - . • '- , ■ ■ 

J* :• Keep track of where the applications, are in the / ./ 

,,:'„-,■;. ■■process, at any one tiiae ' 4 ' ' ' /' 

Achieveinent of these objectives depends oh effe^Jtively 

monitoring the production process To accoHtpl ish ykhi & , the 

__J3?^ce«B0k'^'^re^rtXnl~^^^s^^ 

• ; The indvement of applicatiohs through the different 

steps ■ ' ' ■ ■ ' . 

" The^. time for each step or group of steps 

The accuracy of each step monitore^^ from an- overall 
perspective ■, 

/ Measures an^. standards, ^br activities' 'in each' 
production step 

In* receipt conttol i cojuvting the incoming applications and 

^epiiring them for the inix;ial editing are the two' areas of 

<ijOncern. An initial count is^scequired to reconcile >dth the 




1 coiaiit« of appaications at other points in the process. Also, ^ 

I the applications' must be put into the laroper form jso they may be 
easily reviewed for errors. The critical measure ifor this step 
i^ a count of the incming applications at specif 4<= times. 

In^ the case 6t telephone cal|.s> the number completed 
calls -and the accuracy' of the responses are •of cokcem. The time 
a caller is oh hold awaiting assistance sk>uld be min^^ 

VOnce a calle^ is connected to an operator, the caller should 
receive helpful information. From a management perspecti\(e, the 

- measures of this /activity include a count of the phone eal^s that 
are put on hold. For an ew3curacy measure, a selection of types 
of questions should be developed and individuals should cal/L in 
to see if correct answers are given. In addition, to measure the 
ef fectiyeness of the activity, ratios could bp devi^loped of the 
number of phone call a per applicatiqn* This measure would 

applicq^ht/ and provide a prc>je^ion of the number of calls there 
^11 be at different levels of voltiBie jgrt^oce^siiig^^ gpal 
this step i^ to ensure acciuracy; th^ groups mtisi^ be hoS^ 
if tho.appropriatiB treatwsnt is applied. 





The key entry step is the start of the autcsraated system. It 
is-critic^al to co|mt the number of applications going into the ^ . 
system and mesisure the bac?clog at this point, a*s well as* the 
accuracy of the key entry.;' The measures of the backlog in key 
entry should be ,set up at specified intervals in- the processing * 
calendar* JtoJ the lc6y enttfy accuracy should be iiia^sutred^^ 
sample of applications throughout th^ year 
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critical maasmred are the accuracy of~>eihe compute prociBdure and 
Whether or not the machW -edits are flagging the cortect iteras 
in the correct way. Software quality aaaurance deterraines the 
accuracy the cc»ipute cm test cases, •*^ut from a production 
Control point of view, the ^stem must prdduOfi accurate SAIas 
e'ff iciehtly for different voX^jomes of applications.. The nuittoar of 
times particular edits^are use^ ta\ provide important information 
on the quality of the data and >:he^ system should be noted. 

At the SAR production step it Us necessary to check that the 
data transmitted frcxa the ccsnpute af-e accurately transferred to 
the SAR, ^ich must 'be mailed out al^ng with the proper \ - ' 
infonnatlon to the correct applicants ^ Count ^ of SARs being 
produced* and the number of SARs mailed out should be kept, ahdt 
6pot checks should be made on the accuracy of infbrAatibjt on the 
SAR. Pina^lyV this step in the system should be used as a point ,j 
for timing- how long it takes an applicatioh to make it thr 
the system by followii^ a sample of applications thr o\igh t^f {> , 
process' to their 'final' destination. ' V >!.'v' v^' v-'^ 

The correctio.n.B procegta -must dispose' o^" correct ioQ>- i^*''W|'-' V,. 

■ ■ ' ^ ; .... , ■ . ■ .V ■ '$ 

most eacpedient viay and ehsuiriS that the correct data arte, entered 

, ■ ■ • ■ ' • /■ . ' ' .y'":' 

into the ^stera. Sampling Of incoming corrections should ^e' m^e 
to see vsliether the correct data are being used . The '<i^SuWt^ Of^ 
different types of correctiajtts should be tracked for fco^irect^^ 
actions purposes 
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in suRttnary, the? types of infonnation on th« aiffetent 
aspects of the system caji be combined in t5 the same or a amaXl^ 
nraJber of reports. Appendix A contaips a aeries of sample pro- 
duction conliol reports, AtJ;Ae*"^S^8t managerial level, all the 
information^i ted above ieor each step gloould be included. This^ 
'\«?uld give^n overall view of the performance of the system. 
Reporti^ of these various steps can be done by grouping the data 
inti^if f erent" type«^of applications such ad those' elig ible, 
t^o&0 yalidated, and so forth. Whichever viay they eure grouped, 
[he baaic information on- thjftir status should be included. 
Fiscal .Control v---. r ■ 

;■■^J//^Thev■applicat.ic^ a; fiscal control • system- tp the Pell 
• applicSition^^^p^^ systeib appears in/ Xppendix B. To ade- 

,:^ne^ly,:-ea^ of^each>A,., 
■ prc^^e%^'0n:-'^t^P', ,ahdu}.,d'. be- 'thought'-- through ■''ahead''^of'-'t so ' it ,can- 

.^nsurix^ ; t;h^ ' -bariTect 'issues /.aj:;©' ■'address^' thej^ iMuri^oiQr s ; ■ / 
are :outlir|#i^^^ 1^^^^^^ The: aEre^ ^ two 

■jstepsi/''^- 'I'l^^ '.in:' the-'^iii^ei 'aijd ;2V tho'ia^ , 

tb;\i?te, >£|ddre.^sed • .aft'er •ba fic'' 'a^re^erii has, ^beett' ;i;egtolie^^^ :With'' the 
cqniractot* Appendix al^, <tbntaipts''re formats^^^ can be ; 
^^ied,,,tb\'tw)hil^^^ fp,rmats ^icJ^/afe' 
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general the sped iiic 
specific iojsest are 
l;rapk of specific 
control for the 1 
standardized coet 

For any soft\i«ire 




4 



\\^ 

applied ' to ' ^ 
who must keep" 
scaled to 
nagers to have 



shouid 





techniques with' 



ensure the perforroance 
nianual ^nd some^ are' aui 
Appendix* C ^reseril 

their advantagesr and ydisadvaiy:agesV _ The' li^t ''is-^fsxhadstive," and 
not- all teats shou^ necesaariiy be( sused. ' ]|t depends on the type 
of software and Its jaftirpose. Also/ there i^ a tra3e-o£f "between ' 
the 'increased .accuracy of a test ^d its cos^ 

'■For the pell application &3ftw^ should 
Nj Ibe done on the computing' atgprithm' for aiccuracy and* efficiency 
* for larg^ voltimes of data* Also the e^^ c^iec^ 



thoroughly testet| to ensure they produce' _£h«r*t5orrect edit for ,t;,he 



if*-- 



cokTrect condition. Hi* manager^ ^jhould decide witli 'tests ar,e ^he- 

^ ' H ■ ' ' ' ■ - ..' ^ ■ ' ■ ■ ' ' ■ •■ 

most cost-effective and give tl^ deair^ results./ 



Productivity , ■ \ 

The PelJL application|j|>rocessing system contairi'st several '. 



manual j>rocesses that reqt^ire significant amountis '6¥ iabor-.T 
Since labor coats are generally higher relative ^o other \:.ypes of 
costs in a processing syisteitjj, they, must be closely" rdbnitored* 
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The manual processesi must be efficienti in terns of not 
creating to^ttlenoclcs in th^ system, and be accurate. For / ; 
monitoring purposes, the productivity measures can follow the 
structure of the cost contrql measures, since labor costs are 
major costs in the stej^ in vhich they occur. 

, In the Pell application processing system, four possible ^ 
/areaiS may be amenable to productivity measurement. In the 
information- receipt step, .applications must- be taken out of • 

' .:" .--•^^'■■^r^ V"' ■ : ^ v ■; : : ^ ; 

envelopes and put i|itp homogeneous groups for further processing . 

' Also* Applicants Who have pro^l.eras;;^o^^^ check on the ^^tatUs 

of thfeir applications may te3*ephbne the prpcisssbr ^ - In the SAk 
production step, SARs automatically generated mus^ 'be. put into 
envelopes along with the other information kheets^ Hnally, ^if 

-ctiirtectio^ require a letter <it nse^e other type of special 
handling, pe,ople must be used to g.erverate the type of resB<Qnses 
thatVshpuid •be-'sen't:;back' to ^the^a^plicai^^^ 



CX:- measures and standards for pr^^dtsctivity repori 

. ■ ,■ ' . ' ^ ' 'V ■ ■ ' ■ ■ ' ' ■ • , • 

,; ti-ack the. nua^er of jperson-hoi&rs/re<|U$#e«l''t6 comp step.* 

- * ' • , '•, '/ . . 

the accuraoy of their work, and' the nim^ber of transact iona they 

' . . V : ' , •'■hi:'"'' ' , — - 

haiKilef . *" S.tati sties can be generated using produ 
fiscal measures that pro'duce standardized roeas 
relative' to other costs in the system. They 



quickly the Inanual process is performed. Appendix D presents 



N 



. .aampie report foimta .for toiling-. prodjsifctivlty/ 
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Measures by st6p 5 should include/ ^^t^ 
involved in the prqcess by'haurs or co^t and how many tx an s- 
actipns take plac« aiirina a: specifiei |>eViod. Thi| ahould be a 
regular'ly ,repof ted statistic ■ i| - the 'Second, 
accuracy of the procesasfa sl^uid be/itieaa«i;ed on a sajople basis to 
ensure that the process ntilii^£ |he^'^^^^^ compfi>nent ef fici^tly 
without sM<:j?i^icin^ accuracy .^ K 



QC Giixdeli'nes 



Thi s ' ■sect ion ■ .ist^ge'st s.'' 




LCon.troi ,cca^E?q.nent of - th^\ fcfi' :,ij^o,Q contract.:,. Ti«o t^es of 

■guidelines' are con aidosed;!^^^^ :\i '\"'. ' -r J, " x 



for- 

Pell- proce'Sso'r- 



'fic. guidelines f6r fth« Quality 



fpk. measures |ind standards/ for the^gall 

'■pr.<?:cessor; corit)^l|^ ' 

management afystem for* t|i^ 



The;ab<>^e,.,di'aquseipn- ofi mei»|ur-ea and.v..ataridar9B relates ' ..^ 



■s^ecific^illy ' to- t.he, first i^duei; | I.t, ISv^possible, jto dev-elop, ^ ^'^set 
■^:f ;|fxiid^l'ine»-'^fbr ■a('''cjp,;$is'a^4^;^ syst&a "for ■ iM Pell ; processor 



contufac-t*;v''-; Xt jis essential 
systeiptt. and a sy stent of meais^ 
:,. . ', J?to^3'd^t-aQn,.cpn1^rol 
• ■ "Fiscal- control 

t . ' ' ' ' . 

Wftwa^e',.quali'ty ;a 
prl^uct^ 



. ■■■■ ' • ■/ 
the procejssing function have- a 

emi^rti for J 



isurance 



e the contractor should bis given an opportunity to 



develop such a QC sygteti/ p^A vsho 
g lines of requirement 



I. 



•v;.v' 
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d identi^fy upecific g/uid'e-. 



, The discussion just 



I; » 



given and the technical appendices provide ^ 'general tjack* 
groun4 for specifying the typei^ of mea^ dure 6 the cpntractor shouid 
develop as part of the QC syst&a.' Appendix E displays the raini- 
mtan requirement OS'^A should require for effective monitorij^ of- 

the Pell contract . ' , » . * • » , 

• • ...... ^ • * . . - 

Another aspect of QC fox^ th^ Pell jwrocessor contract- is the 

■■,*■', '."^t '■• ■• ■ V ■ ■ •■' • ■ , •■ . • . ■ ~ 

task scheduling: and cost control System, used, as part of the . 

^ pto ject managements plan. Extremely large production *con tracts, 

lilce the Pell processing contrAct, benefit fr<»n a systematic 

ajjgroach to these issues. SpecificaliY, the arenas of production 

control, fiscal control. > and productivity. can he improved through 

the lise of a cost scheduling system. The requirements for such a 

* ' ' ' ' ■ ■ . ' '■ 

fltystem should 'include the following Capabilitieii^ 

, • A cost control Gantt chart for each phase of the 

process or function that displays the schedule status 
of activities to he accomplished within the^ Baseline 
Cost Estimate (eCE)> funds, available 



lev Bi 

1 



A cost status report that lists fixed >are3 variable • 
costs -with associated activities i 

A resource anaiy&id report >hich describes* in 
^ dollars, th'e status of funds {budgeted or actual) 
within each cost centei: of the BCE 

A cost-vexsua-tirae plot of bt^g6t-versu%'-obligated 
f^ids for each cost cert ter of the BCE -. 

A iritical path of the activities martagemej^t ne*€v«?r1c 
that routinely reports planned and actual achievement 
of planned project activities ' 



The out,puts' iwould. full^ describe for the OS PA. Project 
Director the status of the processor schedule and budgeted vs. 
actual (obligated) funds, The expenditures, obligation, and 
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real^Iocation of BCE fvmdd by the ooatr actor should be stored in 
€he data base tot such a system, enabling the contractor or OSFA- 
nranager reviewing the project to call it up for a review, - 
analysis^ pr update pf prpject^ status. This type of task»^and 
cost scheduling systean i^ vd.d«ly utilized by the Department of 
Defensa contractors are easily adapted to. the processor function 
,£br 'Pell .^•^ y . ' 

The purpose 4o£ this ch|^pter is to apply the general frame- 

processing function. This chapter identifies^ specific data 

necessary to effectjively manage the contract an^ to ensure the 

cotttraetor is perforping to efficiency. Appendices At B, .and D;- 

contain sample reports for each of t'he critical area 3 in the 

processor functioni in order to provide a coherent picture of 

how these reports can be useful in monitoring the Pell ptroqessihg 

contract, an example of a package of the sample reports for a' 

sample cost center is presented Jn Appendix F . Hhe ihformatiqn ; 

receipt cost center has been used, but this tet* of forme can be 

adopted to other cost centers. - ' • 

:Th^s example shows v*^at kind of information a manager would 

receive fr€» a particular cost center. The package of reports 

on the cost center would be included in* a larger package of 

productitin aiid cost data on the entire proqeoaing function* In — - 

the package of statistics on the coat center are the following 

■'Iclnds'^ of "reports: • *^ ■ ' " ■ ' ] ' " '^"^ 

^ • Production control statistics^-These vnould give an 

overall indic'ktion of the ^activity level of the coat 
center. In addition, the/ accuracy of the particular 
step is displayed as an, indication of how efficient 
this step is in comparisorA to othef^ steps. 



P 



41 Fiscal conttol statitetics---This vgould^ive a breakdown 

• of particular Xindsqf coats incurred in the cost 

' center. Also, the different costs are standardized by 

i.r i a volume indicator . 

• Productivity statisticsf-This wo wl<^ given an indication 
of the efficlertcy of the production step and the 
; aiccuTftcy of the step. 

«i Cost control statistics- -These viould provide a temporal 
' setting for the flow of costs throughout the processing 
year. By Kaving /indicators for how much of the total 
/ ' budgets costs ^iiould be spent by a particular point in 
time, should giv-e an indication of how the actual 
• spending flo^ is proceeding . ' - i • 

.■ ^ ., , , i ^ . • : ■ ■ . . ". \ ■ „ ■ ■ ' , . 

QC SAMPLE ■ • ' ■ . ■ ■ 

. There are several area? of the ,prpc«!8sing function where a 
sample of Applicants would be useful in monitoring the integrity 
of the system. ' Current ef for tjs' in this regard represent, how- 
ever, only a start in the widis! variety of uses fo^^ sajnple* 
Every 500th record is now being pulled 4nd tracked throiigfli, the 
isystem. While this method may be sufficient to. satiafy! some QC 
information requirements it floes, not go very fa| in! the potential 
•uses -Of tKe sample. , , ■ ' . . 

Currently.^ most of the information needed about any a'spect 
of th6 application processing system is obtained by accessing the 
entire application^data base. Applicant profile statistics, man^ 



ag^ent reports, and corrective action reports are done by the 



processor v>feo v^ites a statis^iical program that reada-^^very # 
record in the applicant universe. iETl[e_ tables c\a*rert^ 
are, by/and large, specified at the begifeiihg of" the ct^htract ^and 
prOg^'am^ied ift. as part of the ^yst^. AdCessiitg the entire , 
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progC9iaraed in as" po^rt of the system, Accegising the entire 
applicant population lias the following aisadvant^ges.. Such a . 
process Is r . ^ ' *i ' ' ' 

• Expensive- f "The cost of producing a ; table is directly 
v ^ related to the number of records read by the 

^ • • computer . • ^ , 

• Inflexible~ ^The policy process often changes during 

t the middle of the processing year. If a specific t | 

stable is desired in the middle of the year, a 
; , negotiation with the . processor is necessary to 
specially prog t am the table. «^ * | 

• Unnecessaily Detailed- "Osing the entire applicant 
population can ^ g ive deta il abcmt the applicants down 

to any specified level. Much of the .detail may not be . 
/ ah8plutely necj33§a£y_fQr the purposes o^ the .ED 
. managers. 




For ED managers, using the -jitpplicjiiit population seep tx> be ax?, 
unnecessary expense for the gains of added detail awa automatic . 
production. Insteaid* a statistically valid sai|a|>i# of apprica|its 



could be used to provide the necessary statistics at a reasonable 

■ft sample (or s2impl^|s) of applicants can easily be used to 

:■ / / ■ ^ " ^ ■' ■■ , _ ^ " ■ , ■ ■ ' 

perform the fol'lowif^ functions 8 



• ' Applicant statistics 



\m ■ Corrective actions analysis ' 
e i Management reports • * " 

Thfe current set of applicant profile statistics produjjed ^ 

quarterly by the processor, provide a wealth of data about the 

./] ■ . . . ' : ' ■ ^ , ■ ; 
1 character ijti^^^ the api ^l leant .populat io n. The tables are 



useful in their exhaustive de^tail on al^ aspects of appl4.cants. 
Howver^ at^ times the, appropriate table does not exist* The 

■ ' ' ■ ' ■• ■ ■■■■ . / • 

formats of the tables are specified at the beginriijicf of the year 
and are difficult to change- Statistics be estimate from the 
existing tables. If a sample of applicants ^«ere available to £39 
Sitafff the^a|i>pa:opri9ite stuitist could be produced >dth*a^^^^^ ' , 
reaTOnable degree of accuracy and in^a timely fashiort^ 
gainple could nc^ be used for every request, but it <ii<;^u^^ 
a largi^ »imber of the questions that aris^e thtotighout the^^^^^y^^ 
The sample could be dra\itt at the same tier^^ 

are produced to provide a compatible^ set of d^taV^^^^:^^^^^^^^^^,^ " ^ j*^; 
; Corrective actipri anaXysis. requires irifoi^ 
groups of individual s v*io have specif io character istic^^^^ In- 
depth^ analysis of their characterisiici^'and their ■■•traRsact-ipin's^^^^^ 
within thfe syst an require a data bas<e. tOntalp^ing th^ 
elements. A specialised santple of appiioants stratified b 
desired character ijptics would provide ED *dth the necessary; data' 
to perfiona.the analyses. The-. sfiu^i*ii;ag^i |Jrac(^ure', sihoi^ 
part of the system and be as flexible as possible to 4llow ED ; 
staff to ' choose a particular samj^le o^. applicants- to'":;2ii?aly ze . , ' . , 
Specific managesnent rej^rts could be prod ucjpd by sample for 
short turnaround use^. Since the management ireporta now are 

■ -v .. r ■ ' . • •■/^''"^ ■■■ 

specified early in ..^r^e ^roce^s, changed are difficult to produce 
in a short - time f^aiae^kiid. .at..: mjjnjmat,. texpenise . Ins tP 'f ^d ^ ../.a. -aaiaple....... 



of applicants eould be used to produce the reports that, although 

I. ■^^ \ •'■*;" ^ -•^■v 

they voxjld be samples, represent the entire population with a 
reasonable degree of accuTfiey. ^ ■ 
pees pf a Sample • ^y:'.'-' ■ .■. 

Vli^^^^^^^^ raOnjl tor the Pell Grant application process, 

daiartiiat^^C^^ be used to assess the performance of the system' are' 
■ tf:e<iUit^ f - T^^ ■ , ' ; 

• .'Data entry; -^rror • ■ - ^ 

• ; Effeetiveh^ssdf edits and validation 



•S- 



r Av 



^ . tiuch of tiiis information is either una>S|^ilable or 
extt^amely i<?6st^^ to obtain on a program-wide basis* 

\ isajnple sajuple^) • '^^^^ /be uae^ to select case» on Which to 
per^^^ Pirom the analysis done orf the 

gample%v inferences ^ncepj^ing ttirnartfurid time ^ data entry errori 
efj^ectiye^^ qnd ^it etc * # tor the entire jpro^ 




rv gramsfsan^ thieVi '■b^ drawri',.,;, 

^■i;- In d samples.; described, previously,. . 

■...'A samjplinig:^ teeh^^ " • , 

' j''\>in&naa^t,l^n.' ab0iit^\th^^^^ ^n<5 subpopuiatipns. ■ 

, CuVt^rttly, this inf^l^ passing the entire 

, applicant Vfile pi 'pro- 
-grams^: v ■ *<uch,;' of-.tMis;^ data ; could , ' howe^^er'i ', b,«?' .g^erated-,, more • 

' eitfitfieiitlji^' --aj^P olj ■■ac6\iraGV#' ^'.^ing a''3tat,ia'- 



ticajv ^ara^Jie'/ ^T^iF information be obtained using a . * A? 



pralile 's^ni3P?-.e" l^^^c^-'^efi^ s^'^i^ 
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1 



• Total humbejry^f a|ipMcants, transactions, eligibXes, 
ineligiiblea, and rejeb^ions - 

' ■ ■ ■ ■ ■ • ■ ' ■^>" ^ . V ' : ' ■■ V- ■ ■ 

• Average nufliber of transactions per student 

■ ■ > ■ . i , , ■ ■ • " ' ■ • j ' \ :■ - ' 

• - Number of t.ransactionis' by nisober of 
\^v'V-'- - eligible/ ineligible , 

\. Number of applications per atate , ■ 

\;- • - Application -^t^ti Sties \by State ^ , : ■ - , : 

\ • Number of transactions for validated 8t«5ents 

^ ■ • • Number of eligible/ineligible students by validation 
; ' Status - , ■/'" • V " 

• '■ I Drawii^ the Sample .y\ ; ' , 

With any sample coipes a. trade-off- be|. ween accuracy and 

cost. Increasing the ssunple ^size increases the level of acc 

and coat, vtaile decreasing the sample size hae the opposite 

^ /effectii Depending upon the purposes of the Sample i the. optimal 

\ trad«-off between accuracy and cost will dcc^r att different 

. .'sarfiple ' sizen* ' , ■ ^ 

Profile sampi,es ciin be large and t^ieire fore mo^re acc3^ur ate 

because- the** fearginal/cost of ' increasing\th6i' aa is -'^itail. 

In fact, in many instances if* pre ferajble td 'use the\ entire ' 

universe if the need i^r accuracy is great and the coat is 

slight. Alternatively, in' analysis the sample's accuracy raiist be 




■0?R?rifici&d somevaiat because of tine large cosl~^'f~"^€^ in i|0 tHS" 



data^nd analyzing them, "^or^^ eacample, measuring data €»ntry error' 
ij[..a*.ttQAtly and ted iou s prj^cess and gho uld onl y be done on a -1 
small .sample, of cases » . . • ' ' '■' 



„ ■ ■ . ■ 'The; exact ■•^ize of sample ''is- determined ■by\tlte .'expected ■ 
value of the cMracteristics bein^ estiiaated and the desired 
level of confidence. For j^iro file saimples the charlicteriatic/ 
being estuoated is the cell. (A cell ie any category on a tjiU^le* 



For example* in a table of applicants by state, one cell-viould be 
the number, of applicants from" Illinois) . t f tl/<fe smallest ceil to 
be est.imated is less than .1 per sent of the popui^action and a; 95 
percent conf idejuce level la desired , then; t3ie sample slice 
necessary approaches Jthe entire population. Therefore, if a 
large number of categories is being estimated (foi; example ^V t^he 
nximber of farmers per state), a spmple should not be' used . For 
analysis samples, the characteristic xefers to tJ» variable being 
measured t for example, turnaround time ). Piguire 3 presents more 
technical dei^ail for calculating sample size. - 

To. provil^e statistics during the program year , tlie profile 
sample must be select^ed at least once before all the applications 
have been received. One plan vould be to sample, the population 
twice, once in June an<f again in January. This «oul4 correspond 
to the generally bimodel dis.tr ibutibn of applicatic^ns received," ^ 
A random sample «ould contain" the broadest range of infgrmation 
and therefore would probably be most useful for program monitor- 



ing. *The actual selection procedure is, first, to find the sizsi 
and divide thSt by the poj^ui at ion siae to get the sample 
interval. Next, a random start case is chosen- .J'inally/ ev e r y'./:.: 
^th case starting With the ran^dm -start case is pulled, " - 

, 'i' J ' . ' ^ • ' * ,*'''■ 

■M. ■ ' • " 

X • • ■'■79 ■■, ■ - ' 



CHART OF SAMPLE SIZES FOR DIFFEfeBNT VALUES OF P 



I •tet p =» Probability p£ characterlatic of 



Vp m Variance of p + .^02 5 for a two- fai l^d ^ 5% l^.veX of 
■ confidence. .i/.,, 

Fornmla used: \' ^ 



..If. ■.■ 



V 



'it'' 



. oodi . ^ 




.0005 " 




.001 


V 1, 598 > 400 


.005. 




■^pl; ■ .OS.' 


.v'' '■■^■■^^v.,■.l^a./40o;• 


:'.".:05':^ 


- ^••^'^^■■••v'V ,.:' '■^^0,400'' 


.10 


■ 14, 400 


20 ■■■■.r,/: 




:-.uo.-""'" 


..,vi,-, ^^^y::;. ■ m.lt:^} 


.40 


' •^ . / ' ■ tW^oQ 


.50 

■ , - ■; 










X ■ :■ ' ■ ' ■ 1'?^"- . 



^l^ot a profile sainple, .p\ is the expected;; si;^:<5.:.<^^'''j^ ." r- 

frequent^ rcell divfd^d by -'the. total . populat^oh:.>i^?."l^ ■\'^4^^-i'':,.:' :y 

satmple , ""gi -is the expected percent • .figiire .o#.- the -^kri^l^I.e , l?«i"ri'g^''' ^ri^^ 



.iti^a^sured. 




FIGURE, 3 
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. f : 
s .■ 



I.',' 



. -vaieire ;vi: '4:^ the , sample interval . '/ Tlitd ;'se|f&ndyL bef .i! 
"selected In • the : a^ine'.^^y but' u^ng/la ' 'differ^bt/ starting^'cci^e.. v'^'^ ■.'s-' 

The. 'timing' of .the, an^lysi,* s^plfe ^^^pends v^poft; vSi^ther?" th*^^ 

' , ,■.'■■',.' ''■ '■'■A- "' ■•''■""''■■ '';,/ '•■■t^-: '■■"■'"■*/:''''•'■■'''''''• '■.•''.■■ '-Jw'-'' 

/sample. Is tje^ng .ua^, to-,, provide: in^iforrofttioii-oni-t^ 

■/ , ■, , ■ ,.. „ ' ,, ,," 7 

■ |>races8.- fot the cur irent year ' 0trte^dslafi&i^itihti^^^^ ■ ■ f u^^ure' ■ ,yeai-# . 

..■ ' v.-' • ' :~^^'^.$^'^%'^^:i:;:'v^v]''r-^ ■ , c.'----. 

If the, saaple^iS;,usea ; ■ for ,_recoia^ttaation'#'\-|o^^^ years;. theljV :^ 

one aaiaple .-should , be, draw, ■ as ' laite.,lin/"'|h^ '^i^ear, -as ^Js^lfale. . If"' ■. 

■■..'■ v::--';- '.■; i^^'-. / ■: "-:\.^:\ A ■:%■';.■-'•'■'■■■;■::::■%' ■■V'%„vA' '"/'.: ■':':■:: 

the sample i,s used to \provide i^^fprmaMo^ -f^J? ^he curif^nt year . 



,»V'' 



then the sample will have to be djfavs^ duringv t|i year',v and more 
than one seu^ple will probably be deaii'edy pije /pp^ailDillty^^j^ 
be to draw the ^ajnalysis jsaaaple from the prof ile sample. ( sin<;e the 
profile sample will be larger ) . A stktisticai random aampiev 

y ■ , ,' , ■ . , 

stratified to eU. low for broportional repre^eniatioh of the 

•■■■':• , '■-.^■s ; ■'N-^■■'^:;,;,^:^/.,,,.■v■ ' 

v^'-chaTa<iter'istiC9^>being ,: studied,* , wp,uJi.d, b^..- the"' b^st type Qf , sample 

to use^^^^ strati fi<sation y^riablee )^ be MDE pt-ocessor 

^' ;r type, /and: y.alid'ati^n' ^tatti^ ,' ■; .'/- vs/;'^: 4-. "V'-^ -^0,' :,.' 

■ ■ ■ ' V ■ ' ' ■ . '■ . . ' ■ • ■ .. ■' ' 

* ' In c^i|Clu«ion# a^sample or sam^ies could b^ extremely u^teigul 



in |>ravxding' .the in^formation r^equired '- for.'.ve 




toce's^or 



, q,ua-|..|.^,t^./.#.<?n^b^^ profile - sample s'''.e|oluld ,be;,u'sed;. to, generate 'des- 
■fcripiive : ir^forrmatto'n cn,,'- t^ 'appriG^tnt |^pi2lattpn:;;and subpbpu* '^ 
latiojtis ai a reduced cost. Similarlyr analysis samples may 



,:■,<■ ••■•■V ■ 
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■ |»rovide the capability measuf-e th^ v4x^ablee necessary to 

■■ew. ■ ■ '■■■■■' ■ "i ' . ' ' hi ' ■ ' '■ ■ ■■■■ 

monitor, the .system in ^' cos t-ef if icient manner. Ah additional 
>enef it>whicti'",cor^^ ■samprihg'' ■'is:r'tft.|'t:Cll>''^^^ 'have; '"reaSy ' 



'^ccpsa t^ sample a fcr any <juick^ a^^^ statistice desired in 
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SAMPLE PRO^UCTIOS CPtlTROL IIEPORT 
FOR THE PEL L AgPLXCATf ON; PROCESS 



(INfORMATtON Rfi^^EI^J 
■ THIS 



LAST 
PERIOD 



4; 



YSaUl «»f HIS PERIOD 
TO . PATE - • LA6T. YEAft, 



ERIC 



Nuteber d£ Applications 
■Reqeiyed ; - ■ "-''.'v ■ ,„• . ■ 

Numbet" of Applications ' 
Processed ••' , ^ 

v . ^, 

liltjmbfer of Applications 



Rej ected by c|Jf sory 
checks 



edit 



Nianber of Applications 

Nunbe?- of Ap|>ii'Gation^ 
; '■ Backi'oi^^ed, ' ' 

\--;p'^jbce4sin9;;ft,ime' . \' ;■„ ■ " '] 
^■jkeceipt-'to^'Bait^.R^ 

-Applice^tionl ■ t?^*\Key'^ Entry ■ 

■ ] • 

^ 0*2 'days- ■ ^ . ■ ■ . i.> 
■■2-4 days 1 
. 4+ days • •, 

Applications to >©3it\. 
Resolution ' ■ ■' 



^1 



0-2 days * , 
2-4 days 
4+ days 

Acciiracy of Processing 

Sample Size ^ 



- ■ n 



A-l 



-82 





. ■ 








' . v ■ 







1 

'1 



- ■«»■ . 




^ ( IMFORMATIOIS i&EIPT) (Cont . ) 



THIS " LAST 
PERIOD PEJRiOD 



NunOaef Of* Errors 
Percent of E£:ror« 
Types of Errors 



Number of Phone Calls 
Received" 

^Ivirribfer of Phone Calls 
Corapletecl ' - 

Ratio of Calls to 
Applications 



itracy of Process 

Sample Size 
Number of Errors. 
Percent of Ertoi;^ 



V 



YEAR THIS PERIOD 
TO DATE \ ljhST>*J(EMi 




\ 



/ 



^ ■ 
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A- 2 
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APPENDIX A (Cont.) 

SAMpjE PRODUCTIOK CONTROL. REPORT : 
FOR THE PEI^ APPLXGATION, PROCESS 



(KE^ ENTRY) 



THIS LAST YEAR TO 
PERIOD PERIOD DATE 



Number of Appli<iations 
'Entere<3 



Accuracy o f Proems a ing 
_ Sample" Siaa 



: Number of Keyatrpjce Errors 
* Percent ^f Errors' 

;. Number of Data Element^ EVrpr^ 

.■■■Percent.' of Errors^' v V '"'^ '„,\, 

>. : \ > ■ ■ 

■ Number of Form Errors 
- . percent of Er^rorsv 

ljumber 6f Appiicat-ions " 
Backlogged . 




tpiS PERIOD 
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\ 
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A-S 
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/ Nuirtber of. ''Applications 

• Processed (transactions) 

fcluniber of Applications . 
with SAIs 

Nuriii^er Eligible 
Number ineligible 
$ Number Re j ectecS ( no SAI ) 

Nuinber of Transact ibn^L 
/ Ber Applicsint ; (percefctr\ 

• ; one T^-ansactipn 

Two Transactions 
Three Trahsactions - • 
Four or ^tore Transactions 

♦ Number of Applications with Erfrors 

-Number wiMi,l-| Errors 
Number wixn 4+ ^rtfor's 

Types o£. Errors 



•A. 
B 

C . 
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THIS I'ERIOI 
^S" YEi 



■ -v. 



r 



^APPENDIX ^ (CoriJ. . ) 
SAMPLE PRpmiCTION CONTROL REPORT 



'R( 

FOR THE f ELL APPLICATION PROC|:SS- " 



I^AR PRODUdtlOH) 



^ Jliinibeit of SARs Produce^ 

tilumbdr ,of SARs Mailed ... 

Accuracy of Prbceaaing 

. Sample size 
Nutaber with ^rror 
Percent with error 
Types of error 

■■ . .. • A 



THIS lA^T YEAR TO 
PERIOD ^ PERIOD ' DATE 



THIS PERIOD 
> LAST YEAR 



I., 



.Pi^oceasing Time' 

Scrapie Siae 

Time of Processing 

^ 0-10 days 

10-15 days 
15-20 days 
V 20-25 days 
25-30 days, 
30r40 days 
40+ days *" 



f 
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> ;\ppENDIX A (Cqnt. ) 

y ■ ■ • , 

SAMPLE PRODUCTION? CQOT^RQL REPORT 
FOR THE PELL APPLICATION PROCESS 



(CORRECTIONS PROCEDURES) • 



A' 



THIS ^LAST 
PERIOD PERIOD 



YEAR TO 
DATE 



THIS PERIOD 
LAST YEAR 



Number of 'Applications 
with Corrections ^ - 

Percent of /^plications 
with Corrections 

Processing Time ^ \ 

Corrections prolliassed 

0-5 days 
5- 10 days 
10-15 day$ 
'15-20 days 
20+ days 

Numl?er of Corrections 

Transaction^ Per Applicant 



1- 3 

2- 4 
4+ 

Accuracy of Process 





4i 



Sample Sise 
ffumber with Errors 
Percent with Errors, 

Number of^ Corrections 

Backiogged 
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APPSNDIX.B 



APPLICA TIOM Of FISCAL CONTROLS 'VP ^HE PEI^L PROCESSOR CONYRAGT 

(STEP i) 

STKP KtCY ISSUKS , COMMENTS 



(Btift)re Piractiss 
• Stilt ts) 



IdwriLify Major 
"Cost Centers** 



IfJentify specific 
Costs (What) 



Du tUimine 
MelJKK]olcxjy (How) 



V 



Type of contract I 



If not Cl>Ft% the level of detail des- 
<?ribed balow raay not be approi>riate. 



Which parts of con-- 
^V^qt need - 
fiscal mbnltpringf;: 



I nclude al 1 '*cost canltets** y that 
have ptoveit difficult! to control in 
current contract and /those vhich 
project to exceed 5 £:|ercent of j 
total cost • . 



/ 



Probably includes •*SER productii., ^ 
•*data entry ^ " correspondence , •* ^tc * 



Miicjh costs incur- 
red ne^d fiscal 
liionitoring i , ' 



^ Sincie the major ct>8ts in t?iia con-- 
tract are labor ar^ comiliiter time# 
thQgte should be the otiXj costs 
inonltore<%. 



Most efficien;t I 
metho^l of fiscal 

loonitorirHj 

\ ■ . 



Use **loa<ied** labor co|3ts and direct 
computer time coj^ts. * 



Ms tabi 1 shm^nt of 
Tolerance Lievels 
{How Much) 



# Level must" produce 
effective *'early 
warniiyj* system" / 



3pecf fy Guide t ines 
and Require Contractor 
Kesipon^e 



. Ob t a in iiFi*j contractor 
resixinGe 



# 10 j^ercent to. b^ applied on a* cost 
center level* ♦ 



Specify no response to fiikral con^ 

trol Bystem wii:|. m4k.e technical XV 
nonre&pdnsive • 



1 :^ 




APPB^lblJi B (C6NT. ) 



: APPHCATIOM OF FISCAL CONTROLS TO THE PELL PROCESSOR CONtKAfil^ 



KEY ISSUKS ' 

« Ability of ppntractor 
to iinplfeiueiit fiscal 
coritirol systeia for 
contraut 



tXfiiliiat^ flesfponses 

■■■■ . f ■■ ^' 



■ ■ COMMENTS ^ ■■■ 

- resfx^nsiveness to g 

.experience pn otiieri contracts 
; ,\i nnovn t iv ^ . ^tiggf e s t ibn9 ^ ■ . ■ 



Mailt fy Guidelines 
(I f ,Appli,cable> 



; At tl\i.s st.ag|^ bo«sjder it*nbvativ| 

I suggestipnis ^nd- raodify;' auial^iin jointly... v'"'-.-- 



r 




■7; ■■■ 
. '>•■; 



♦ I 



0 




m 
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V Appendix b (gont.) % 

APPLXtATiON OF FISCAL CONTROI, TO THR PEti • PROCE^SOI? GQK2?RACt 



STEP 

D<2 V e £ dp ^'1 ex ib I e 
'Uutlyet dased on 
Vol Ui»0 



(STEP 2.) 

KEY vISSUES' 



. C6MMEIITS 



W 



Seg r ega fcinfjp ** f ik:^ 
and •^var iabX<ef 
coste; • 



"/'/•"^ft, -■;:,#;;^,.^bc:^wl^'.^■be;:a6l^ a Joint 



Mechanisms 



4ppirbf>r ia kje f oriiia t 



\;CI^,t& •■o'eB^^T: : derail: ijr 



OH Guidelines with « 
Wiimlng Contractor 



JJ ■ ■ '^-^ ''"^'X'l' ■ ' 'v^ '•■ V ..'-. •- K'.'i ^ " "jii::. 



OS FA Monitoring Systen^ puM^JAne 



92 



4^j^^:ne&!j<,tb'.J^*'^^ 



Variable Budget 
; Volume Indicator 
SAlRs BrbdticJ^d -> 

■ ■ * i_ . ■ 

: Variable Costs *^ 

Labor Cost "Cat a 
. -fintry 



fed 
I 



YC^f At/ VAEt ABLfi / 
COSTS. ' 



/'i 



Fixed Costs 

Labbf Costs: Data 
Ent^ry* Supervisors 

. m-ML FIXj])D COSTS 
TOVi\L COST CENTERS 



. bSFA dONTRftCTOR 4^1 SC AF., CONTROL 

(SAMPLfe.BEPORT) (EXCEPTION REPORT) 
PBRXOb ENpiljG 10/15/82 



CURRENT-PERIOD, 



: BUDGET 



999^ 



99999 



99999 



1*99999 



A CTUAL 

/ » ' .it 

9999 

■ "' .■•» ■', ■ 

^9999. 

■ ttr-- — T'V 
9999^ ' 

999^39^ 



99999 



99999 



99999 



99999 



99999 
99999 



C0HllH«IltSJ 



*■ ■ 



PRIOR PERIOD 



BOOQET 



99999 



99999 



999999 



ACTUAI. 



99^99 



^9999 
99999' 



99999 



99999 
99999 




99999 
9999^ 



detaileil re|K3rt for each cost cent;jer with costs in excess of b<iti^^t bjf greater 
than 10 pertJent slKJuld be sent to the OSFiV pro-lect officer alor^ witli the.mahag^tiiant 

... report. . ' „■ . ^ ■ ■ -'f /^.^^ 
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The contractor Project Manager shcmld foliowup with a i;5aau#l explanatory report 
withjin ofte week {cdpied* to oo--site ^cmitQr, also whether enpec: ted or reoccur ]^ . 

/ ■ ' , ^ ' ■ . . .. . ' ■ • * - ' ^ 

Thij^ report should flay individual line it erne out of range. > 
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bSm CONTRACTOR PISCAL CONTROL 

j(Sample Uofigart) iBxceptioo Repcfrt) 
Period' Landing 10/15/82 



CURBfeNT PERIOD 



V 



PRIOR PERIOD 



CUMUIATIVE 



VH i icT| pl en iidg e t? « 
VQlmtve I nd leg tor . 

^Computer T4wie 

*» . . 

i..4bpr Cost. 

Data ERtry' ^ ^ 

TOTAL VARIABLE 

Fixed Costs 

Labor Costs 8 Data 
Entry Supervisors 

T OTAL FT XKD XQSTS 

TOTAIi COST CENTERS 



•BUDGET _ 

9999 , 
# 99^99 

999999 

99999 

■99999 
99999 



ACTUAL 



9999 



BUDGET 



9999 



ACTUAL BUDGET 



9999 



99999 



ACTUAL ^. > 



99999 



"4* 




99999' 


99999 


999999 


999999 


99999 


99999 


« 0. 




99999 


99999 


99999 


99999 




* 



» 99999 '99999^ 



999999 9999999 



'999999 



.9999999 



99999 999999 



99999 999999 



999999 



999999 



SI9999 . 999999 999999 



"-/pomtnentss 



Ebr small cost centers (pr ao^ 11 contracts) aosts ' Bhpul<^ 'be captu?-erl on an 

ihdivldu^l. ei^ployee Y^asis. ' * ♦ , 

■.'■■■■». ■ • ^ ^ ' ' Ip • , : , ; ■ ■ „ ' ■ „ ■ ,; , ■ 

This repf>rt sliould be fudtvlshdS within one we e|c of the end of the period to onrsite 
monl'tor* * ' ^ - ' * * - 

■ \ . . ■ ' . v, , ■ ^ ■ ■'■ ; ' : ' :\" \ . * ^ .•■ 

This forwat^ enables OSFA to quickly coa^r^ ao^aal costs to bodget ar^'prldr 



wc^ths 
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Us* *of variable b^dgel: indicator presents a key to possible cost vatiances. No 
costs vary directly to indicator t>ut follow trend. ' ^ . 
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Cor I c sipondos(ice ' 
SAR *ProductAon 

'ro l'Al r (!OtlfTH ACT L 



OSFA CONTUACt'OR flSGAJ. CONTROL 

— ■ ' ■ — ■ ■— - V — • ' 

^ Period Endi^cr^^0/XS/82^ 



GBHRENT PERIOD 



PRIOR PERlbD 



Budget 



99999 



' 99999 



A«tual Pudge t* 
9999 9999 



99999 


99999 


' — lr^. 


> 


9999f> 


99^999 



Aptual Budget 
^ 9999 9999 
99999 999999 



CIWUTJVTIVK 

''Actual 



9999 



999999 



22221 ^99999 * ^ . 999999 



Coiiurt'cntss' 



• . This repor^t should be furnished w^thirt one week to 'the OSPA Prdject Officer 

• . pip of these reports (tor each month I should be easily reconciled to the montlilY 

m A flag could b^ inserted wh^ife tolerance level is exceeded denoting exception 
. teport ro follow. ' ♦ ' ^ - 



> '^^^ ^^^^ Projedt Officer should sunsnarize ififoriSation to OSPA manageiaent . 



I 



SOFTWARE QUALITY CONTROL TECHfcllQUfi COMPARISON MATRIX 



'SOFTWARE VERIFICATION TECHNIQUES / 



TEGHNIQUE 



Drivers 



Test Data 
Bases 



' PESCRlPTt ON/ PURPOSE 

Superfluous ^ C0de needed to 
perfoi^Bi unit testing and 

t Ion it?^ bot t oin--up so f tw«i rfe 
development effort 



<1 ' * 

Collection of data stprfed 
on computer pei^ipheral 
d^^rice which closely 
matches real" dat# base 



- ADVANYAGES . 

• i) Testing of critical 
components ban t« 

■ -emphasized'*' v,' 

2) l/p funotipns aire most 
often provided early in 
r software development > and 
these i:an easily be han*^ ' 
died' with driver progr^rna* 



1) Real* data base ia^ 
protected* 

2) Testing can proceed 
faater: wi th lesa , - . 
confusion. 



> OISADVAHTAGES 

1) CoBt of develdpmf^nt qM^ 
of drivers which are J 
usual ly di«;^rded at end of 
teat;ing - ^ -^^ " • - , . ' 

2) Errors in drivers may 

impede testing ^ - 

♦ ■ ■ . ' . - '■ ' . . ' 

' ' ' .A . * ■ . 

3) privets usually written 
by satBie prograpners wh6 
writB coii^nenfa, being. * 
tested, and &o may contaia 
same invalid asBuinpt ions > 

1) Higb cost ' 

2) May, not contain a<3e-- 
qitate data types or euf fi- 
cif?ntr vol\iine of da ta . . 



Design 

Verification 



Exarntini^tion or inspect 
of a software specif i, 
tiion £br purpose of f 
ing design errors . 




1 ) Error ireworlc epst 
decreased doe to early 
error detection^* 

2) Quality 6£ product is 

iwpro^^ed. * . - 



1) Often considered 
tadiou^ arkd boring 

2) Some errors thus found 
^ight be more easily found 
by cKSiJiipiler after ^cidirtg , 
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lox 




TECHNIQUE 

©cut ion 
a lysis 



Automated 
Network or 
Path 



Statistical 
Prediction 



' APPEIsIDIflC C (qonti ) ' ' ' • 

SOFTWARE QUALITY CONTRDU TECHNIQUE COM PARISOti MftTRIX 

, — , , — ' ■■ - — ; — T'TTT^ — — — "\ — 

SOFTWARE VERIFICATION TEGHNli3UES 



PESCRIPTtON/PURPQSE 

Automat^ed raoiiltoring of 
computer tbaseii eof tw^re 
t^s t tng t i V i t ies , qol-: 
laeting dta f row thijgt 
testing , awalyzittg data 
manually^ arid then predic- 
ting duratiofi' aiKi co&t of 
testing' 9n4 quality of 
software product 

Examination *af source co4e 
to deterpiine mi tm\im aet 
of paths vii^loii^ exercise 
all logical brarithes pf a 
prograiii 



Coinputatlo^ri of a confi- 
dence factor which indi- 
cates ef^fectivene^spf 
prograttwiifig ^ 



' ADVAm*AGES • 

■, ■■' ' ■ ■ '' * ■ * 

1) Automating control .and 
^onttcoring t^st activities 

2jy , Aid irt ert^^ 
granuning ' and sysjtem deve- 
lopment st&ndardsy 



1) Automatic deterwlna*- 
tion of optima^ paths to 
.be tested 

2) Aid in enforcing pro- 
gramming and control 
structure stand'arda 



1) Will let programmers 
know with' eoiae ijer^ainty 
that an error .exists 



I) High cost 



1^ Large number of itera- 
tiona reijuired may be 
excessive for contain/ types 
of software ^ ' % 

2) Need for user interac-- 
tAve to identify iniompati-* 
ble brancii ^pressiotpis ' 

i) bi££icuii to deterniine 
t/pe Of number- of errors to 
Vbe inserted . ' 



I' 



APtaoiX C (Cont . ) * 



SOFTWARE tjUALITY CONTROL TECHHIQUB COMPARISON MATRIX 



f ' 



SOFTWARE VERIFICATION TECHNIQUES 



TECHgl^JE 

FisnctlQnal 

■.Testiiig.;,,^ 



o 
I" 



PESCtClPT ION/ PURPOSE 

Execution of independent 
t tests desigif»'i^d to demon- ' 
strate a speci f ic f unc; 
tional capability^of a/^ - 
program or software system 



1 ) TejBting is yisx 
oriented to usei; 
evai uat ion ^ ^ * 




and 



2) Punctiorts completed 
are me^asurabiei r 

3) Testing of functions^ 
applies to ^11 phases^of , 
test activities « 



DISADVANTAGES 



1) All ^eci^ioa points 6£ 
a progra«a\ not ^n^P^searily ^ 
tested vV..'".'^. 

2) Dej^end^ent upon good 
priginAl fiinctionai 
specifications , ' 



4 



Design 
Validation 



t 



ISxarain^tion pjr inspeetlon 
of the functional require- 
ments. ^SJid^thje. desxg a . .Oif. a ^ 
software system for the 
purpose of finding errors 
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4) Revision and control 
of't^st specification is 
vsiQplifie(^*^ 

ly Error reworK coat is 
decreased due to early 
erron. dfitefitloa* . , . _ . 

2) Quality of product is 
imprpve/l. , ^ 

3> indreasa in programmer 
productivity is due to 
positive psyeholc^ical 
' effect!* ' '.' ; „. 



^ IK Ofteo considered 
tedious boring 



'4 



10§' 



APP^DiK C t C^t , ) 



SOFTWARE QUALITY CONTROL TECHNIQUE COMtAEISON MATRIX 



SOFTWARE VERIFICATipN TECHNIQUES 



TECtiNIQUE , DESCRII*S?iON/ PURPOSE 



"advantages 



DISADVANTAGES • 

- iatriSc^ ^ Us9 of a formB-priented" V 1) Gives complete andL^ I) /'Ito efficient of 

!naly9^i6 aord language or a fprraalV syn- ; ynambiguqus *^ta atrucftura analyarng problem ddfinl- 
I rol^lem ; t:^x language ta^ "def irii-tiona / ^^'^ — 

<l ftatement ca/te^needs of t^rkp^^^uBex tp . . v. 

'2) Provides precise 
^f^iodule dependency ititer'-^ • 
, £ace definitions • 



lan^uag^s 



the analyst 



tiona 
2) 'High cost 



rai^c 



3 ) Assuram^e of proper 
control Jpgic to sequence . 
software tasKs ' c^r^ctly 
for all input cbiin>inations 



Af * Assures that; i^ach taa^ 
, kB ^functionally <^oinpre1^ 
/and unambj|^pusly ^^fimed 
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APPENDIX Cr (CQl\jt.) ;^ 

SO^,TWA'RlS QUALITY CON TROL TECHNIQUE COMP ARISON MATRIX 
. SOFTWARE VERIFICATION TECHNIQUES 



TECUNIQUE 



ADVANTAGES 



Litnit ChacKB A series of compafe * 1) tleducea impact of * 

inetructor^ .gt^tidg i^t » ^^rrore . 
- V. conditions and keyed to a ^ 

table within memory con-^ 2) Provides simplified 

taining hicfh and iow par^- ^ technique to control 



DISADVANTAGES 



Record 
CouBt$ 



n ' 
t 




4' 
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soeters of thp lii^it 

Automated co^ir of records 
or documents ; ■ 



&)£tensive^ pr coiaplex 
except ions 

3) Focusea only on 
except iofiiB « / 

4) Easy to implement and 
understand * \, * - 



i j . Allows errors to be 
undetected v 

2) Requires ffequent table 
changes in' » dynaioic , 
eixviromaent . ! 

3) May hide out:<)f -bsilanga 
condition 

4) -Frovides lesis control , 
as'^he range o^ accept|ible 
values' is jsltpaiided « 

sy .Unscheduled ch^ngjes can 
causd ^majority of transac-- 
tions J|:o^ be riejected v - 



1 ) ^Economical 

2) Easy, to design aftd 
ii^lei^^nt 



^> Functional value is^ 
dependent upon the correc- 
^ /^pn and reinput procedures 
for re j^ied dataSt - ' r 

I ) Does not identi £y lost * 
documents 

I 21 Opes not iridicat;e pro7 
' cessing accuracy , 
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SOFTWARE QUAL ITY CONTROL T EC HNIQUE COMPARISON MATRIX 

\ ' • - • . ••■ . • -v. 

SOFTWARE VERIFICATION TECHNIQUES 



TECHNIQUE 



DESCR I PTION /PURPOSE 



V 



Batch Totals* CouritiTig of nuinbar af ^ 
and Countd batches and records within 

a batch', a batch being a 
' - ^ group of documentB/recordis 
of eaine type compiled into 
proceseing aequenee and 
assigned a unique batch 
. ... . .1. -Tiai^j^er' ■ - • . ' / 



I 



3) Isolaites problibiti 
are.a(sj 

4) Provides quiclc check 
for docaiwent/ irecoird loaa 



ly Indicates procefsifig 
accuracy 

2) Ejstablisrhes a 
record-value correlation 



3) SimpUfies 
-reconciliations , 



DISADVANTACES 

3) Requires manual recon- 
ciliation and corrective 
actions 



4) Reconciliatiorm are 
difficult. 

. ■ * ■ ,■ ' ' ' " 

5) Does itot ensure 
locating missing documents 

1) ttot effective with hig)f 
volumes . 

2) Time conauroing to 
prepare aned reconcile 

3) , Requires fUequent I 
—-ad^ustifljents- for errors — - 

4) Curobetsonie wi,th online 
■ ptodee'aes ■ ■ ■ 

. I. ■ ' ■ , 

5) Decreased yalue in 
distributive systems 
context 4^ 



lio 



lii 



APPENDIX G (Cont .) 



SOFTWARE 6UALI-yy CONTROL TECHNIQUE COMPARISON 
SOFTWARE VERIFrCATIOt} TECHNIQUES 



MATRIX 



* TECHNIQUE 

. .. ; ♦ r — — r-i— — . 



Totals 



» . 



o 
I 

•>4 



hash Totals 



ERIC 



M2 



DESCRIPTION/ PURPOSE ' 

A total of sigiiif leant 
data^ computed either' 
nianually or by machine, 's 
from, document /records to 
j^ro&eased 



A tota^ computed on Inslg-* 
nificant dat& Mike SSN) . 
used to cor^t%oX docun^nt 



1) 'Indicates procesel^ng 
accuracy ; 

2) Indicates ratatlve 
valtae of errdrs ; 

3) Creat^^aa a l^yprbdu t 
of processing . ^ 




Indicates loss 



2) May idelitify lost 
<^cumeikt ^ 

3) May indicate errorsj in 
input preparationa 

4} Easy to^ create 



/ PISAPVANTAGES \. - * 

1} - Usually requires manual 
: prepaitatlon h • 

^2) Difflciiilt to maintain 

■ ,« / ' ' ■ 

3) Does not identify. 
Causes erf errot 

4) Does not indi.ca,te num- 
ber factors 

5) Difficult to apply to 
online propessing - , 

6) Curoborsomei cpatly, and 
complex ifi distributive v 
j^ydtems t 

1 ) Reb|uird|ii manual' inter- 
venition 

2\ Dofe» ^t iaentlfy erraij"'" 
Sjlgnifica^e 

3) Difficult, to r^oncile- 

if auraetroua errora occur; 

•* ■ ■ . . ■ ' ' ■ .1 ' 

4) AwlMtfardiy handled by 
onlihe processing v * 

, 5 ) Difficult to imp lenient 
ill shaite4 data bases* and J[2 



APPENDIX C (Ccmt.) 



SOFTWARE QUALITY CONTROL TECHNIQUE COMPARISON MATRIX 
SOFTWARE VERIFICATION TECHNIQUES 



TECHN IQUE 

f ... 



n 
I 

. 09 



DESCRI PTIOK/PURPOSE 

lised to idefttify data/ 
records contained on tape 
or disV 



ADVAt4TAGES 

'■ . .. ■ . ' ■ ■ ' ' ■ ^ ■ 

* 

1) Positive file • 
idea tlf ieat ioEi 

■■■■■ 

2) Inliibits accidental 
file! purges or updates 

3) Provides documentation 
o£ files used 



DISAPVANTAGES % 

1) vMay beignp^€d or- 
bypassed ' 

2) Not geared to* online 
data bases 

3 ) Stequlires human inter-- 
vention to enter identifi* 
cation in£ona£it ion 



I 



o 
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• ■ ' . • APPENDIX C (Cent. ) ^ 

. ■ : HSE :PR POTEefriAL USE VERXFICATION AND VhLtmHim TECHNIQUgS 



AND 

-VALlEATIOtI TECHNIQUES 


V ' SOFTWARE 


DEVELOPMENT PHASES 


TWt? Tiff Tnf* T/M^ 




TMPT.PMirKfT^ilXT TnM 




Drivers 


• ■ ■ . ; 

f 






■ • ^ ■ . ■ ; 

f 


Test Data Bases * 
Design Verification 


















Execution Analyslsv 








, • X 


Automated Ne tworlc of 










statistical Prediction 






■ ' . ; ■ ■•■x> ■ 




Spinet ionat Teisting 








,'; .;V' ■.■■■X . ,,■ 


Oesicrn Validation 

* * .' • ' . ' ■' . 








■ *■ . ' 


Matrix Analysis; and 
Prpbteni Statement 
lianguages - 


1 








Top-Down PrograsMBing' 









.A 
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SAMPLE fiROPyCTIVITY REPORTS FQR> THE ♦ 
PELL APPLICATION PROCESS <TIME PERIOD) 



Nmabfer of Ap^ications 
Processe^j' 

Number of Hour* Worked 
Number of Persdmiel 



( APPLICATION RECE; PT ) 



PERIOD 



LAST 
PERIOD 



\ YEAR 
TO DATE 



Type 1 
Type 2 



TotJil Labor cost ' 

Unit Time CoS^ (Hour/App) 

Unit Labor Cost (?/Appy 

Accuracy of Prftcessiii^ . 

Sample Size • 
Number wi'th Errors 
Percent with Errors 
Type of Error s v 

• A 

■ B . ' • ■ , 

■ C 



Peak Backlog 
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APPENDIX b (Cortt,) 



r 



SAMPLE PRODUCTIVITY REPORTS FOR TH B 
*f PELL APPLICATION PROCESS { TIHe PERIOD } 



(APPLICATION RECEIPT)' 



Number of Talephone Calls 
NiBjiber of i^urs 5)6rke<? 
Nurnb)^ of persorinel 
* ■ Type 1 . ■■' ' 



THIS 
f»ERIOD 



LAST 
PERIOD 



f YEAR 
TO DATE 



V 



/ 



Nujitber W Chills completed 

Total L^ibor Cost 

Unit Labor Cost <$/oall) 

llfccuracy of Process t»g ^ 

Cajinpie size 

Niffl^er of Errors ^ 

■percent "s^f. ■Errors ■ ' ' 

Number of Phone Galls -per 
Transaction 



Peak Bacldog 



Ik 
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* " APPENDIX P (Cont.) 

SAMPLE PRODUCTIVITy REPORTS FOR TH^H^ 
PELL APPLICATIOlit PROCESS (TIMS'^PlRloS' ) 



(APPLICATION R 




' THIJ 
P] 



Number of sARs Produced 
Bruraber of SARs Mailed 
Nuinber of Hours Worked 

Niimber of Personnel 

• ■ . ■» ■ 

Type 1 * 



Total tabor . co st : , ; ' '■, , , 

Unit Lai3or cost ($^/SAR) 

Onit -Tim<i "cost fhour/SAl) 

Accuracy of Processing 

Saiftple size 
Number of Errors' 
Percent' of Errors 

Ntanber of SARs per Transaction 

Peak Backlog 



' LAST YEAir 
PSRlOt^ TO DATE 

■ ■ \J 



7 



( 



i 
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^ APPENDIX P (Conttl ' * 

• SAMPLE PRODUCTIVITY REPORTS FOR THE 

PEbL APFLXCATION PROCESS (TIME PSR]^ ) - 

PLICATION -RECEIPT) ' 



... ^ 




THIS LAST YEAR 

^ PERIOD , ' PiaRIOP TO DATE 



Number of corrections Processed 

■\/Typ6''-A , '> ' ' ". ' ,. ■ 
Type B- ' • 



Number of Ho'ur s Worked 
NuiJiber of Per aoonel 
Type 1 

' -..Type. 2 „: ; ,; 



Total x«abor^2oat 
Unit Labor Coat' t$/cOrrection) 
Unit Tiitue Cost (?U5ur /correction) 
Accuracy of^ Processing • 
Xyp^ A A 

Sample Si 2ie _ 
Number of _ Errors 
Percent of Errors : ' 



r 



Sample Sise 



Corrections per Transaction ^ , 
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^ liifori^t^i^^ Receipt 



I 



Ciirsory Edit 
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APPENDIX K ' 



REQUIREMENTS FOR A'^P'ROCESSOR QC SYSTEtf 



MEASU RE /STAtfDARP 



I 



• > Count 



Backlog 



Count 



# Count (edits 



Accuracy 



EXPtANATION 



The nuraber of appliqations fieed to 
be counted at the start iag point • 
ill the process to use as a control 
tbtal for reconciling other &t©ps 
of the* process « ' , 

/ ■ >!, 
SiiiG^. this ts a tirae-eonsuniliag 
and labor, Intensive proceae this 
nieasure would give an indication 
of tlie efficieticy of the pro-, 
(luotion step. 

The^ application^ roudt fcte segregated 
by the typd of treatment so a count 
of th^ applications thftt are ■ f lagg^ 
by an edit must^ toe' made to Keep track 
of the applicatioiij,;^ 

The number of times a particular edit 
is usedl prbyides information Oh tlie 
applicationvfbrfa data items. 

TShe segregation of applications itttb 
*their proper ^rdup ^uuet l?e done 
accurately t(S avoid rework and bottle- 
necking in t»e system. Prej^ ring the! 
dppl'ication for e*ttry. into the syetretra 
is important to th6 efficient func- 
t|,c»ning of the system* 



<-■•. " 



7 



122 



APPEtiPIX E (Cont.) 
MINIMUM REOUIREMENTS'FOR A PROCESSOR QQ SYSTEM 



r 



AREA 



MEASURE/STANDARD 



EXPLANATION 



Key entry 



Accuracy 



BaeklQ9 



Compute 



Count 



Tbia is thd pdlnt %^er6 data are 
entered into tliiiif 8y&teii\ and prQcefiisdd> 
A count at this point is needed to act 
as a control on the automatic pro- 
cessing st.ej^v 

* This ia a n^jor nieasiy^^einent to itiake 
because the inaccufacy of the' dat:a in 
the ^ye tern requires rewoi^k and a ioss 
of inteigrity of the Entire delivery 
ayatem. ^ > ' | 

this atep is a. potential l)ottle«eak 
since da tat are entered mantxally. The 
backlog is a measure of the efficiency 
of the step and the use of latx>3r In ^ 
tVie step. ' . '"-J 

•Another control total is taken at thtis 
Step to ensure that there is no loss 

7"of ■ applications'." ■ ~ ~ — , 



!■ • Count ( edit ) 



The number q£ times an edit is used 
must be kept* "Etack of order to 
evaluate the quality of the data ^n 
the system. Corrective action 
analysis can be done from this infor- 
tion. . 



APPENDIX E <Conti.) 



MINIMUM REQUIREMENTS FOR A PRCX^ESSQR QC SYSTEM 



MEASURE/STANDARD 



E>Ct»LANATION 



SAR Proauction 



Count 



Corrections 



3 



• Accuracy 



# Timeliness 



Count 



Accuracy 



This provi^e^. a contra 1 total on th^ 
prodessin^ jayet^m and another count 
of t^e number of envelopeet being . 
mailed out ad^ a xtontrol total on^tbe 
whole., systefii..' ' ' - ^ '^z ' 

The inforniatiGn»on the SAR qoea back 
to the applicai^t and muat re|iect the 
data cixiiisig into th© ^yatem^ 

This 'point is v/here the prQcess ends* 
Tptal tuFiiarourid time of the ptocdisft . 
ia mpaciurecl ^t this point* 

The types pf 'Coif;r 6c tiops being ^ 
received should be tracked to * 
evaluate ' the, quality of the data . 

The entering of the daita must be don^ 
accurately to sAsura sioboth and timdly 
~pr oces S"i.n9 • ' ^, t~; — " ~* — ~" ■ — - — ^" ■ '"^ , ~- 
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APPENDIX E (Cont .) 
MINIMUM REQUlREMBr^TS FOR A PROCESSOR QC SYSTEM \ 



AREA > 



MEASURE/STANDAaP 



EXPLANATION 



Component |Vc66racy 



'^ut^syst|id accuracy 



System /Accuracy 



# Efficiency 

• Test Aa^di 



Bach cotqpon&nt sliouXdi . 
be testree^ to. ^neujce that", 
it worica ititerBal'ly . ; 

• ' * ^ ■ ' 

ofhe cofaiponents that fit j 
together must work together 
in order to produce tlie 
corr6qt ^xei^ults* 

' ■ * ' ' ■ . ■ 

The subsysUsms must be put 

together iato a total operating 

syeteni that produces the * 

correct result. < . 

The jBystein must produce the 
desired reaults in aft ef ficienf^ 
manner in order to keep down 
co^tis and minlTKtse backlog. 



As a part of accuracy and 
effici^ncyj t^est data must be 
used to check all parts of the 
sy«teiii to tfe^ if th« correct 
results are b$lhfc| produced and 
the systera can process large 
ainounts . of data.* 



APPENDIX E ( Cont . ) 



MINIMUM REQUIREMENTS *FOR A PROCESSOR <JC SYSTEM 

— ■ J _ . , . : 



MEASURE/STANDARD , 



EXPLANATION 



Cost centeirs 
♦ Tolerances* 



Volume indicators^ 



ill Eixed/Var ia):>le Costs 



Repprting i^ecTianism 



The cost centers of the contract 
must be i^eil;tified and a method 
for monitor ilngf costs within the 
center developed. 

Development of a method for What 
costs should he measitir^d and 



they slhould hQ measured* 



Tolerance ranges for types of 
costs shcmid be d^terwine^ in 
order to produce an early warning 
system far monitoring* ' 

The costs vary with volume i and 
so the unit of volume fihoul^ be 
determined to standardise the 
CQ6t level's » 

The categorizatlbn of costs naeds 
to be done to determine the * level 
of •Gostlsi by leveils of volime* 

Costs ehouid be reiported in 
^imilar^ ways for. similar :^nc- 
tions^ The same cK^st categories 
and the same levels at \i^ich" 
costs are accumulated 
should be, reports^ similarly^ 



. A- 
AREA 



Productivity 



I 

0t 



MINIMUM REQUIREMEtiTS FOR A PROCESSOR QC SYSTEM 



MEASURE/STAtJDARD 



liGvel of effort 



Volume, indicatoj 



Accuracy 



EXPLANATION 



The level of effort. * in nUiaber 

of hours or people should a 

measure of the efficiency of 

the system. This laeasure' can ' 

be broken ^ut by labor categories ^ 

and coats aasign^ to th^ to ^et 

labor costs by cost centers. 

This is used in the determination 
of the efficiency of the •production 
step by staniiardiair^ costs. ^ 

The efficiency of the Step is 
Daeasured ift jpaft by th6 accuracy 
of t^he %#ork performed. Building 
this roeaeure into the productivity 
measure will evaluate ,Uie net 
efficiency of the. step. 



erJc 



1.31 
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. SAMPLE FISCAL CONTROL REPORT 
FOR TJfE PELL APPLICATION PROCESS 
C I IIFORMATI OS RECEIPT) 
(period ea4iag ) 



NujiOser of AppIxcat;ions Received 

^ isfvaaber of Applications Procesaied 

Number of Applications 
Rejected Cviraory Eflit Checics 

. Number of Applications 
Going to Key Entry 

Ntimber of Applications BaOtLpg^ed 

Processin9 Time 

(Receipt to ESdit Resolution or 
Key Entry) ■ ■ ■ ' 

Applications to Key Entry 



THIS ^ LAST . 
PERIOD PERIOD 



SXEAR ^THIS PERIOD 
TO DATE LAST YEAR 



/ 



0*2 daji's 
2-4 days 
4 days 



Applications to Scl it Resolution 

0-2 days • 
^ 2-4 days 
4+ • days^ , 

Accuracy of Processing 

-Sample Size 



^1 . 



i33 



F-1 



■ f 



APjPESpIX F (Cdnt.) 



, SAMPLE FISCAL CONTROL REPORT 
FOR THfi PELL APPLICATIOJ* PROCESS 
.(INFORMATION RECEIPT) 



V 



VARIABLE COSTS 



Autc»raat€i(|' Cost<9 



Labor Coots 
. Total Variable Costa 
FIXED COSTS 

r piauit. Material 
Labor Costa * 
fotal Fixed Costs 
Total Cast 



4* 



CURRENT PERIOD 
BUDGET ACTUAL 



PRIOIt PER?OP • 
BUDGET , ACTUAL 



Number of Applications 
Processed^ 



of 



k Unit Vari^ie^Cost ($/app) 
Unit Iwib;b.r\Coat {$/app) , 



C: 



Unit PixWd ^ost ( $7^PP> 



. . ' ' :^/un^^;^tal . ■^^^ ■ 



1: 
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APPENDIK F {Cont, r 



FOR 



S.AMPyLJ££G«. CONTROL REPORT 
,• TS#PMrr^PLiCATIOl!l PROCESS 

(INPORMM^N RK:EIPT) 
(period eading ... . ..■ 



Coat Category 



Budget Actual 



% of Total 
Budget Amount 
Should be Spent 



% of Total 
. Budget Amount 
Actually Spent 



Total Variable Cost 
Total Labor Costs" 
Total Fixed Coat 
T^tal Cost 



1^ 



i 



SAMPLE FISCAL CONTROL REPORT 
FOR THE PELL APPLICATiptI PROCESS 
(INFORMATION RECEIPT) 
(period ending ♦ ? 



Number of Applications 
Processed > 

■' ' ♦ 

Number of Hours Worked 

Number of Personnel 

Type 1 
. Type 2 



THIS 
PERIOD 



LAST 
PERIOD 



Total Labor Cost i - 

Unit Tii!W? Cost (Hour/App) 

Urfit' Labor Cost (.$/App) 

Accuracy of Proce/sln^ 

Sample Size 
Number vith Errors 
Percent ■ MdLTb Errors,.- 
. ' Ty^- of Errors ' ' 

A •• '.- .1 : 



